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Preface & Abstract- Around the spring of 1974, my major professor, John P. Basart, came back from a 
Radio Astronomy conference where he had heard exciting news of a new approach to spectral estimation 
and computation. We discussed it at length and agreed that I should make it my PhD research topic. I had 
the background necessary to do research that was both theoretical and with practical applications. As I 
began the study of this ‘new’ topic, I realized that there was a general need to study a broader topic that 
included several methods that could be practical for sampled data from systems that inherently added 
noise to the measured signal. 

Throughout my research, I considered a complete development of the Maximum Entropy Method 
(MEM) my biggest challenge. I soon learned about the work of John Parker Burg, then at Stanford, and I 
tried to develop a collaborative relationship with him but he rejected my offer on the grounds that he was 
doing proprietary research for an oil exploration company. Due to many circumstances, including my 
decision to leave academia in 1976 for work on the new Global Positioning System, I never contacted 
him again. 

I was also unable to determine what papers he had published or the availability of his thesis until 
my work was nearly completed so I did not benefit from his insight. 1 could not find a source for copies 
of his early work but managed to find information that he and produced XX papers and XX presentations 
on MEM. They are included in the bibliography for completeness but I did not have access to them. 
Frankly, I forgot about my MEM work for almost 40 years and finally decided to Google for his 
dissertation and found it. He published it and a book in 1975. MEM was a very hot topic in the late 1970s 
and 1980s and I missed out on the fun. I remember reading on the NSF that they would no longer fund 
MEM research so don’t bother sending any more proposals! 

This dissertation covers both the theory and practice of estimating the spectrum of signals in 
noise using digital data. The theory of describing some of the signal processing concepts for digital data 
are given and various spectral estimation methods are given. The theory of MEM is described in detail 
using approaches from estimation theory, communication theory, and statistics. The work was intended 
to give researchers the theory and practice of practical means of spectral estimation using 
communications or scientific data. The Maximum Entropy Method by John Parker Burg is explained 
from what was known in 1974-75. 
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PREFACE 

Although there are copious publications available that address the 
topic of spectral analysis in general and many that address time series 
data in particular, it is difficult to find information on how to apply 
the general theory to practical data processing. User oriented applica- 
tion techniques and computer programs are seldom published and canned 
routines save the researcher uneasy as to their biasing effect on the 
results. This research was done in an effort to provide a complete system 
of applied spectral analysis. Special attention was given to describing 
the effects of measurement and sampling error, observer bias, and 
interpretation of results. 

Much of the text is basically tutorial and, as such, is not original 
but the organization and presentation are unique. Except for Chapter XIX, 
the original contributions are discussed in the introduction. The author 
is especially happy with Chapter XIX because it represents several ideas 
on interpretation he believes to be unique. To the fullest extent possible, 
authors of works already published have been gratefully acknowledged by 
the many references. 

The author's major professor, John Basart, has contributed greatly to 
this work through his helpful technical discussions, objective criticism of 
important ideas, and diligent proofreading. His original interest in the 
Maximum Entropy Technique and conviction that it would provide an important 
dissertation topic are the embryos of this work. It could not have been 
completed without his continuing encouragement and patience. 



The author's very special thanks go to his wife, Kathy, for her con- 
stant encouragement that gave him the desire to finish a difficult task 
and to his daughters, Carla and Tracy, who spent many of their growing years 
with a "student" father. 

The author is indebted to Betty Carter for typing the manuscript and 
for her excellent formatting of many difficult equations. 

Readers interested in these topics are encouraged to contact the 
author concerning the ideas presented not only to discuss their merits but 
also to generate new and better approaches. 


Ames, Iowa 
January, 1978 


Steve F . R ussell 
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I . INTRODUCTION 

The analysis of empirical data for the purpose of determining funda- 
mental parameters for a physical system is a basic goal for much applied 
research. One of the most important characteristics to be determined is 
the frequency spectrum of the recorded data. Spectral analysis plays an 
important role in many scientific fields such as radio astronomy, meteo- 
rology, geophysics, mechanics, statistics, communication signal processing, 
econometrics, physics, and biology. Probably the most important research 
applications of spectral analysis are presently in the fields of geophysics 
and astronomy. The most advanced work on the spectral analysis of noisy 
empirical data is being published by geophysicists in connection with oil 
exploration geology. The search for new sources of oil in deeper domes 
and offshore has placed severe requirements on the available methods of 
data processing. To improve the spectral resolution of the measured spec- 
trum, these researchers have developed many sophisticated techniques with 
the most advanced being maximum entropy spectral analysis. 

Maximum entropy spectral analysis (mem) has stirred the interest of 
radio astronomers because its claims for better spectral resolution with 
limited amounts of data creates a promise of better sky brightness maps for 
radio sources. However, both the theory and application of mem are not well- 
developed and, consequently, this tool is not available to most researchers. 
Initially, the goal for this research was to develop the theory and prac- 
tical implementation of the maximum entropy spectral analysis method for 
estimating the power spectral density function when the autocorrelation 
data are incomplete. As the research progressed it became apparent that 
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many problems and barriers existed between theory and application of spec- 
tral analysis. These problems were further investigated and it was con- 
cluded that a more -general approach to spectral analysis should be taken. 
This approach led to a more user oriented thesis. Such topics as estima- 
tion theory, frequency domain analysis, system analysis techniques, sam- 
pling theory, and Fourier analysis were investigated. The selection of a 
method of spectral estimation as well as the mathematical behavior of the 
most common methods was developed. 

The author's original contributions are a collection of smaller topics 
scattered throughout the text. Chapter x, the Hilbert space vector for- 
mulation of digital Fourier transform analysis, is independent work that 
constitutes the mathematical framework used to develop the spectral mixing 
formula discussed in Chapter XI. The discovery of a companion function to 
the time autocorrelation function is original work and is discussed in 
Chapter XV. The mathematical development of the calculus of variations 
technique used for this derivation was developed independently and is 
given in Chapter XXI, Part J . Much of the development of digital Fourier 
transform spectral estimation given in Chapter XII is unique to this text. 
The author's approach is one of analysis and interpretation while most 
published material concentrates on the computational details of the fast 
Fourier transform ( F F I ') . The chapter on methods of spectral analysis is 
original work designed to aid researchers in selecting appropriate esti- 
mation techniques and to help in understanding the problems conmon to all 
spectral estimators. Although the topics of sampling theory presented in 
Chapter VI are not new, the presentation and derivations are specifically 
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designed to relate the effects of sampling to spectral analysis. The dis- 
cussion of MEM is not original but the computer program and interpretation 
of results have not been previously published. 

Much work remains to be done in this research area. More theoretical 
study of MEM is needed to determine optimum series lengths and more effi- 
cient methods of spectral calculation. Data reduction schemes and better 
interpretation of estimated spectra need further research work. Probably 
the most needed future research is on the problem of unequally spaced data 
samples and undersampled data. 

Most of the mathematics and terminology used in the text are those 
commonly used in current publications in the spectral analysis area. One 
of the author's more fundamental contributions has been to unite the ter- 
minology used by geophysicists, statisticians, and electrical engineers to 
provide a common basis for communication, where more than one terminology 
exists for a given concept, every effort has been made to define and use 
the varied terminology. The term ti me function is used to denote a contin- 
uous function of time that can be completely described in mathematical 
terms. A time seri es is a sampled or discrete time function and may be 
ideal or empirical. The term data function is used by the author to denote 
either a time function or time series that is being numerically processed. 
The author defines collective parameters as those characteristics of a data 
function that are used to'describe the process that generated the data. 

An asterisk is used to denote complex conjugation, a script F operator 
denotes Fourier transformation ( ^{x(t) } =X (w», and convolution is abbre- 
viated by the product; y(t) =x(t)*h(t). 
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This research and resulting thesis are most directly suited for use by 
the researcher who mist do very serious spectral studies. The casual user 
is better served by applying readily available "canned" approaches to spec- 
tral analysis. This distinction and the possible alternatives available 
are discussed in Section D of Chapter IX. 

It will become apparent to the reader doing a detailed study of this 
work that the techniques of autoregressive spectral estimation, maximum 
entropy spectral estimation, and recursive digital filtering all have 
si mi I ar mathematical forms. They do share a common mathematical and 
theoretical basis but they produce different spectral estimates because 
they employ different criteria-of-goodness and numerical algorithms. These 
simi larities were useful in helping to better define the properties of 
these estimators. 

It must be emphasized that this work has a wide range of applicability. 
It is useful for any empirical work involving the Foarier transform or the 
autocorrelation function. The variables of time and frequency are used for 
convenience but others such as space and momentum are equally valid. All 
the methods of analysis presented involve some type of model fitting. In 
estimation theory, a truly "parameter free" method is not possible. The 
selection of a model is the most important task in spectral analysis. 

Chapter IX was written to help clarify the selection of a model and the 
computing method. 
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II. TIME AND FREQUENCY CHARACTEIUZATION OF TIME 
FUNCTIONS AND SERIES 

A time function or time series can always be specified by enumerating 
all possible function values in the time domain. For the continuous time 
function, this is done by representing the function by a mathematical equa- 
tion. For the discrete time series, the entire description can be given by 
an autoregressive equation or difference equation. This complete descrip- 
tion often involves mathematical completeness in the sense that the math- 
ematical representation is valid over the entire time domain and is well 
behaved. In many scientific applications, this complete description is not 
necessary. Quite often it is sufficient to specify more collective param- 
eters such as average value, mean-square value, or power spectrum. Such 
collective descriptions often give valuable insight into the physical nature 
of the process which generated the observed data. For this reason it is 
desirable to discuss some of these collective characterizations. 

For theoretical purposes, a time function or time series can be con- 
sidered as being a sample function of a real process. This process can 
generate a time function which has both periodic components and random 
components. Often a statistical description involving probability density 
functions and higher order moments is considered suitable. For most cir- 
cumstances, the observable in this system is some time function or series 
and it is the analysis of this time data that must be used to study the 
process. The definition of. many of the collective parameters of a real proc- 
ess involve the use of statistical definitions. To be able to obtain these 
same parameters from time data it is usually necessary to assume that the 
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process is ergodic. With this assumption it is possible to avoid the use 
of statistical descriptions of the process and obtain the collective pa- 
rameters directly from the time data. Most of the theory that is presented 
in this dissertation will be concerned with time functions that are sample 
functions of real ergodic processes. The use of probability descriptions 
will usually be limited to those that are necessary for a better theoret- 
ical understanding. None of the actual signal processing techniques will 
involve the direct use of any probabilistic descriptions. 

In the time domain, a data function can be characterized by its time 
average value, mean-square value, and variance. These collective parameters 
can be related to the statistical description of the function and yet do not 
require any transformations or use of statistical parameters. The time do- 
main analysis of a data function can also involve the computation of the 
time autocorrelation function (Appendix I, Part E). The time autocorrela- 
tion function is a measure of how well the function correlates with itself 
in time and also can be used to determine the mean-square value of the func- 
tion and detect any periodic components of the process. 

In the frequency domain, a data function can be characterized by its 
amplitude spectrum or .its power spectrum. The amplitude spectrum is ob- 
tained by taking the Fourier transform of the data function. The existence 
of an amplitude spectrum depends upon the existence of the Fourier trans- 
form. In many cases, the amplitude spectrum of a random process does not 
exist. The power spectral density function for a data function is obtained 
by taking the Fourier transform of the autocorrelation function. This def- 
inition is superior to the use of the amplitude spectrum because it elim- 
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inates the problem of existence. The power spectrum of a data function is 
often the desired physical information that is needed to solve a problem or 
verify some scientific hypothesis. For discrete data the power spectrum 
can be determined from the discrete form of the autocorrelation function or 
by using the autoregressive power spectrum. 

In theoretical discussions of the characterization of data functions, 
it is possible to give a mathematical formulation which describes the func- 
tion over the entire domain of the independent variable. When the data 
function is obtained from a physical process, the mathematical description 
becomes only an esti mate for the "true" function. The various collective 
parameters which are used to characterize the data function are derived 
with the infinite domain assumption. Measured data can be described by 
collective parameters only when mathematical estimates of the entire data 
.function are assumed. For example, a finite time series can be represented 
by a discrete Fourier series or by an autoregressive series. Both of these 
representations assume that the data function can be defined for all time. 

In Fourier analysis this is referred to as the periodic extension assump- 
tion. 

Once a mathematical description of the data function is obtained, the 
various collective parameters such as average value, mean-square value and 
power spectrum can be calculated from the theory. As a consequence, these 
representations are only valid for the infinite domain assumption. It is 
important to realize that a concept such as power spectrum is totally de- 
pendent on the mathematical definition and is not related directly to a 
physical concept. The mathematical definitions for the various collective 
parameters are given in Chapter IV. 
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III. OBSERVABLES AND ESTIMATION THEORY 

The mathematical description of an area of physical theory such as 
quantum mechanics or spectral analysis often involves the use of transfor- 
mations into other domains, vrhen working with this theory, it is important 
to realize that quantities defined in domains other than the time domain do 
not have a real physical significance and hence are not physical observ- 
ables. Quantities such as amplitude spectra or power spectra depend upon 
a mathematical definition for their existence. The definition is usually 
chosen so that it helps to give additional useful information about the 
physical process that is described by the function. As an example, for a 
periodic function, the amplitude spectrum obtained by taking the Fourier 
transform gives the complex amplitude coefficients of the Fourier series 
which describe the time behavior of the function. 

A time function can be defined for all time but a real observable can 
be defined only for a finite time. This causes problems with definitions 
which involve the use of infinite time functions. Transform domain func- 
tions such as amplitude and power spectra are defined only for functions 
which exist for all time. These definitions must be modified to accommodate 
the "finiteness" of a real observable. The modification is usually designed 
so that it gives a good approximation of the "true" value. Another approach 
is to use mathematical models to approximate the observed function with a 
mathematical function which is defined for all time. The study of how to 
formulate mathematical functions which describe data that is obtained 
empirically is called estimation theory. 



9 


The simplest forms of estimation theory are concerned with the selec- 
tion of an approximating function to "fit" a set of empirical data and the 
selection of a criterion to judge the "goodness" of that fit. A function 
or computational scheme that is used to describe some property of the empir- 
ical data is referred to as an esti mator. Estimators can be as simple as a 
single sine function or as complicated as a digital Fourier transform. The 
three estimation processes most commonly used are those of signal filtering 
or smoothing, prediction or extrapolation, and interpolation. These pro- 
cesses are typically used in conjunction with a desired mathematical process 
such as numerical intergration or Fourier transformation. The estimation 
processes of smoothing, extrapolation, and interpolation are all useful in 
spectral analysis. 

Estimation theory is used to help solve the common problems of numer- 
ical integration, difference equations, and the representation of empirical 
data by exact functions. These methods are discussed in books on estima- 
tion'theory and numerical analysis. A partial list of these methods would 
include for interpolation; Lagrange interpolation, Hermite interpolation, 
divided differences, Newton's forward and backward differences, Gaussian 
interpolation, Stirling's interpolation, and methods due to Bessel, 

Everett, and Steffenson, and for numerical integration; Riemann summation, 
Cote's formulas, Simpson's rule, Romberg's method, Filon's formula, Gauss- 
Sidel method, Gauss-L aguerre method, Gauss-Hermite method, Chebyshev's 
formulas, and aid-of differences, and finally for curve-fitting of empir- 
ical data; least-squares polynomial, orthogonal polynomials, trigonometric 
functions (Fourier series), exponential functions, Chebyshev polynomials, 
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continued fractions, and rational functions. 

Why are there so many approximation methods? The reason is that es- 
timation theory cannot specify a unique method of estimation for every 
class of problem. All of the listed methods incorporate assumptions about 
the measured data that make that estimate optimum in some sense. For 
every criterion of goodness, a slightly different estimator can be deter- 
mined. 

Estimation methods often derive their name from the criterion used in 
their selection as "optimum". Such schemes as the method-of-least-squares, 
the method-of-moments, and the method-of-maximum-likelihood are often seen 
in the literature. Estimators can also be classified according to such 
characteristics as linearity, bias, sufficiency, efficiency, and asymptotic 
efficiency. 

The selection of an estimate as "best" is strictly a matter of user 
definition. Experience with the use of practical estimators has shown 
that certain criteria produce good results for a variety of tasks. One of 
the most popular criterion-of-goodness is to minimize the least-square 
error between data values and estimated values. The sum given in Equation 
3-1 is often referred to as the total mean-square error or "residue" of 
the estimate: 

1 N A 2 

mean-square error =- Z (x - x ) (3-1) 

N n=l n n 

The data set is represented by the x^ values and the estimations by the 
val ues. 
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Estimation theory is an important part of the theory of spectral 
analysis of empirical data because of the finite observations involved. 
Estimation theory can be applied to truncated functions to provide the 
"best" possible spectral estimate for a given amount of data. Criteria of 
goodness for spectral estimators are often not well-defined and an attempt 
will be made to discuss this problem for each of the various estimation 
schemes. 

A rigorous background in estimation theory is not necessary for the 
study of spectral analysis but it does help to clarify the goals of the 
various schemes and puts the selection of a particular scheme on a ration- 
al basis. For the reader who is interested in a good introductory text on 
estimation theory, the book by Deutsch (1965) is recommended. 
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IV. TIME AND FREQUENCY DOMAIN ANALYSIS 

The time and frequency domain analysis of a time function or time 
series can be done on a theoretical basis or on an empirical basis. The 
theory for this analysis has been well worked out and is widely published 
in communication theory books and statistics books. The empirical analysis 
of a data function involves the use of estimation theory and criteria of 
goodness and is not well defined for the user. Empirical analysis also 
involves many subjective decisions and a priori assumptions about the 
process which generated the data. These factors make it easy for the 
observor to bias and influence the computed results. This has been espe- 
cially true in spectral analysis where the experimenter often must choose 
among several techniques which give dissimilar results. In this chapter 
we will attempt to summarize the theory and equations that are used to 
characterize a data function and make special emphasis on the differences 
between theoretical definitions and estimations. 

A. Theoretical Definitions 

A data function is completely described by an equation or recursive 
relation which specifies all values in the time domain. Although this com- 
plete description is adequate, it is often more meaningful to describe the 
data function by such collective parameters as average (mean or expected 
value) value, mean-square value, variance, autocorrelation function, am- 
plitude spectrum, and power spectrum. A discussion of the theoretical def- 
initions for these parameters will now be given. It will be assumed that 
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the data function is defined for all time and is well-behaved so that there 
are no mathematical problems with the definition. The theoretical defi- 
nitions will depend on a knowledge of the probability density function for 
the process and on the assumed ability to define the function over the 
entire domain of the independent variable. Most of the theory to be given 
can be applied to deterministic functions as well as random time functions. 
The discussion will emphasize the random aspects of the data function but 
it will be assumed that the reader will see the analogy for deterministic 
functions. 

The expectation (or average) value for a random variable is defined 
as: 

E(x} = f x p(x)dx (4A.I) 

all X 

This theoretical definition depends on a complete knowledge of the prob- 
ability density function p(x) and the existence of the integral. 

For a periodic function, the average value for one Fourier period is 
computed from the integral: 

1 MI/2 

x ( t) =- 5 x(t)dt (4A.2) 

T -T/2 

The mean-square value for a random variable is defined as the second 
moment i ntegral : 

e{x 2 } = J x 2 p(x)dx (4A.3) 

all x 

For a periodic function, the mean-square value for one Fourier 


period is: 
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1 i+T/2 2 

x (t) ? I x (t)dt (4A.4) 

T -T/2 

The square root of this value is called the root-mean-square value of the 
periodic function. The mean-square value for a random variable or 
periodic time function is often called the "power" of the function because 
it is a measure of the second moment and is used to calculate the power 
in an electrical circuit. 

The variance of a random function is a measure of the variability of 
the values of the function. The variance is defined as: 

= J (x -E{x}) 2 p(x)dx (M .5) 

ail x 

The variance is often referred to as the "ac power" in the function be- 
cause it is a measure of the variability after removal of the average or 
"de" part. An equivalent definition for the periodic function could be 
developed but is rarely used. Quite often the average value is subtracted 
from the total function and a "zero average" function is defined. This is 
done explicitly in the definition of the variance. A good example of 
this approach is the use of a zero-mean autocorrelation to define a power 
spectrum. Blaclanan and Tukey (1958) use the term autocovariance for this 
zero-mean function. Other descriptive parameters such as higher order 
moments could be defined but these are not often used. This completes 
the discussion of time descriptions and we now go to the autocorrelation 
function. 

The time autocorrelation function for a data function is defined by 
the following multiplication and integration process: 
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I 

R (T) = LLm O x(t)x(t+r)dt (4A.6) 

x t-® t -T/2 

The autocorrelation function has two very important uses. First, it can 
be used to describe the "coherence" of a function because it is a measure 
of how well the data function correlates with itself after a shift in time. 
The process of shift-multi ply-average emphasizes periodic features in the 
data funccion and is useful for determining any periodic structure in a 
data function which would normally appear to be stochastic in nature. 
Second, it is used to define the power spectrum of a stochastic process. 

The W iener-K hinechine relations (Appendix I, Part F) define the power 
spectrum of a random function as the Fourier transform of its autocvrrela- 
tion function. 

We refer to the autocorrelation function as the representation of the 
time function in the "autocorrelation domain." The domain variable t is 
called the lag time or simply the lag. The lag time is the amount of shift 
between the two functions. The autocorrelation function cannot be used to 
derive a unique time function because of the integration process. This 
means that the autocorrelation function is not unique with respect to the 
time function. For any given autocorrelation function, an infinite number 
of time functions could have been used to derive it. 

The frequency domain descriptions for a data function are limited to 
the amplitude spectral function, the power spectral density function, and 
the autoregressive spectral function. Other spectral representations such 
as using a filter transfer function are valuable for special applications 
but are really just special cases of the three general methods. 
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The amplitude spectral function is defined as the Fourier transform 
of the data function: 

X ( w) = i5x(t)e-j ^dt (4A.7) 

The amplitude spectrum will exist only when the Fourier transform exists. 

It is most applicable for periodic functions and single pulses in the time 
domain. It also forms the basis for discrete Fourier transform analysis of 
sampled data functions where the function is assumed periodic beyond the 
sampling interval. 

The power spectral density function is defined as the Fourier trans- 
form of the autocorrelation function. These two functions form a Fourier 
transform pair as given by the W iener-K hinchine relations: 

-ko 

S x (ad = J R x (T)e ’ Ja3TdT (4A.8) 

-00 

-H* 

V T) = h J s x (u,)e+jun<itu (4A - 9) 

—CD 

In theory, a knowledge of either function implies a complete knowledge of 
the other. In empirical analysis, this assumption is rarely true because 
of finite data. Consequently, the various estimators and algorithms will 
have an important effect on computed results. It is the development of 
the theory describing these various estimators that is the primary goal of 
this research. 
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B. Estimations 


When working with empirical data, the data function is only an esti- 
mate of the "true" function and all collective parameters that are calcu- 
lated from the empirical data set are esti mated parameters. If the data 
set is large and the estimator is well chosen, the estimated parameters 
are very close to the true parameters. For theoretical reasons, and also 
for asthetic reasons, most estimators are chosen so that they are unbiased 
and converge to the true values as the amount of data becomes infinite. 
There are important cases where the estimate that is ^sed does not converge 
to the theoretical value but still gives an acceptable estimate. This oc- 
curs often in commonly used techniques of spectral analysis. The estima- 
tors that will now be presented are widely used in the analysis of empiri- 
cal data. 

The average value for a time series or set of random variable values 
is estimated by the sum 


where the Pk are the discrete probabilities associated with each data 
sample x^. It is commonly assumed that all of the samples, x^> are equally 
likely which means that the Pk are all equal. For N equally likely random 
samples of x, the average value is the simple arithematic average given by 


1 N 

= N k=l*k 


(4B.2) 


where P k = jj 
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For a periodic function, the average is estimated by the integral 
1 AT/2 

x(t) = O x(t)dt (4B.3) 

T -T 12 

where T is called the period of the average. Generally this period is not 
equal to the Fourier period because the Fourier period is unknown. The 
Riemann sum approximation for this integral is given by Equation 4B.2. 

The mean-square value for a discrete random variable is estimated by: 

2 I N 2 

x = S. p fcltVk < 4B ' 4 > 

k=l k 


When all N samples are equally likely, the mean-square value reduces to: 

2 1 ^ 2 

x = - s x . (4B.5) 

N k=l * 

For a periodic function, the mean-square value is estimated by 
2 I <+ T ! 2 2 

x (t) = — O x (t)dt (4B.6) 

1 -TI2 

where, again, T is called the period of the average (or estimate). The 
Riemann sum approximation for this integral is given by Equation 4B.5. 

The variance of a random function is estimated by using a weighted 
average summation approximation for Equation 4A.5. This estimate is: 

A2 _ TJ 2 - 2 

CT X - (x- x) -x - (x) (4B.7) 

When all samples are equally likely, (4B.7) is estimated by the sum: 

*2 _ I N , -2 

°X “N kll (X k- X) 


(4B.8) 
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"The estimation procedures used for autocorrelation functions, auto- 
covariance functions, amplitude spectra, power spectra, and autoregressive 
spectra are the most important aspects of spectral analysis and are the 
main themes of this dissertation. The procedure for selecting a particular 
estimation method and its application to empirical data will be discussed 
in Chapter IX. The details of the estimation algorithms will be presented 
in the appropriate later chapters. 
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V. SYSTEM ANALYSIS TECHNIQUES 

There are several important system theorems and mathematical tools 
used in spectral analysis. Many of these are associated with system 
analysis in electrical engineering and are also broadly applicable to many 
other physical systems. For completeness, and to familiarize the reader 
with many of the concepts to be used later, these analysis techniques will 
now be discussed. 


A. Convolution Theorem 

The time output response, y(t), of a physical system can be related 
to the time input, x(t), by convolving the system impulse response, h ( t) , 
with the input. A simple system is shown in Figure SA-I. For a linear, 


x(t) 


h(t) 


y(t) 


Figure SA-I. A Simple Operating System for Illustrating Convolution 

time-invariant system, the convolution i ntearal relating the input and out- 
put i s w ritten as: 

-H» 

y(t) =. J h(t- T ) X ( T)dT (SA.I) 

-CO 

or in shorthand notation: 
y(t) =h(t) *x(t) 


(5A.2) 
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All functions are assumed to be defined for all time and h(t- T) is the 
impulse response of the system occurring at delay time, t =t. 

The impulse response of a system is defined as the system output 
response when the input is a unit impulse (Dirac delta function). This 
can be shown by using the convolution integral of Equation SA.l and the 
sifting property of the Dirac delta function. 

The impulse response of a system can also be obtained by first com- 
puting the system transfer function, H(w), and taking the inverse Fourier 
transform. It is important to realize that H(ui) and h(t) form a Fourier 
transform pair: 

+® • 

H(ui) = f h(t)e" JU *dt (5A.3) 

_C D 

h (t) = ^ J* ; H(ui)e +jttf: d(D (SA.4) 

—00 

This relationship is frequently used in electrical engineering to help 
analyze and synthesize electrical netvorks and systems. It is often easier 
to synthesize an electrical network in the frequency domain and then use 
the inverse Fourier transform to determine its time response. These equa- 
tions all assume infinite time functions and do not take into effect the 
requirement of causality in the system. 

For h(t) to be causal (no output prior to there being an input), the 
impulse response must be zero prior to zero time; h(t) =0, t <0. Also, if 
the system is to be stable, the impulse response must be absolutely in- 
tegrate (the response must die out as time goes on). The true limits of 
the convolution integral are actually the "overlap" of x(t) and h(t- T). 
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Only for the theoretical case where both exist for all delay time, t, do 
the limits become infinite. At a specific observation time, t =t 1} the 
output will have a value corresponding to the value of the convolution 
i ntegral : 

y(t| ) =i^h(t| - T)x(i-)dT (SA.S) 

a 

A typical graphical interpretation of this convolution integral is shown 
in Figure 5A-2. 



Figure SA-2. A Typical Graphical I nterpretation of Convolution 
Showing Limits of Integration 


For physically realizable systems with causal impulse response, the 
convolution integral is usually written as (Schwartz, 1970, p. 87): 

y(t) = 5h(t- ,-)x(T)dT (SA.6) 


For both x(t) and h(t) causal, the convolution integral relating the 
system output with its causal input and causal impulse response becomes: 


= £h< 


y(t) = Oh(t - T)x(r)dT 
0 


(SA.7) 
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This representation of the system response most closely represents the 
true circumstances when a signal x(t) is applied to a system h(t). The 
system output, yet), can be interpreted as a superposition of past values 
of x(t) weighted by the system impulse response, h(x). A widely used 
graphical interpretation for the convolution of two causal functions is 
given in Figure SA-3. The impulse response h(t- r) represents the true 
system response after it has been "folded over" and extended into past 
time. The system response, yet), at observation time t, is the input 
function, x(t), from r= 0 to T=t weighted by the "folded over" impulse 
response, h(t - t) . 

The convolution integral of Equation SA.7 is used in linear systems 
analysis to develop the Laplace transform method of solving system re- 
sponses (Cheng, 1959, p 227). If x(t) and h(t) are both Laplace trans- 
formable, the Laplace transform of the output is equal to the product of 
the Laplace transforms of x(t) and h(t): 

yes) =X(s)H(s) (SA.8) 

This important result is called the convol ution theorem. 

The convolution of finite time signals or periodic time functions 
with a causal system can be analyzed using the concepts of the amplitude 
transform, both Fourier and Laplace, but the analysis of the response of 
a system to a random time function must be done using autocorrelation 
functions and power spectra. This theory will now be summarized. 
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a) Causal input function 



b) Causal filter response 



L pre& ent 

c) Convolution for causal functions 


Figure 5A-3. Graphical Interpretation for the Convolution of 
Two Causal Functions 
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B. System Response to Random Time Inputs 

A linear, time-invariant system, h(t), operating on a sample time 
function of a random process, x(t), defines a new sample function, yet), 
that is related to the "old" function and the system impulse response by 
the following convolution integral: 


+® 

yet) = J h(u)x(t - u)du 

-CO 


(SB .1 ) 


This integral is equivalent to that given in Equation SA.I and we see that 
the concept of convolution is valid even for random inputs. This alternate 
form of the convolution integral was chosen to make the following deriva- 
tions simpler. This result is very important because it will facilitate 
the derivation of important relationships between the input and output 
sample functions. 

If we assume the input process is both stationary and ergodic, the 
time autocorrelation function can be used to determine the power spectra 
of input and output. The time autocorrelation function for the output is 
defined as: 


1 ^T/2 

R (t) = Mm o _y(t)y(t+ r)dt 


Y T ....* cd 1 -T/2 
The output at time t and at time (t + O can be written in terms of the 
convolution integral of Equation SB. I as 


(SB. 2) 


yet) = ^>h(u)x(t - u)du 
-*> 

y(t+T) = 5h(v)x(t+ t - v)dv 


(SB. 3) 


-CO 


(SB .4) 
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where u and v are dummy variables of integration. If these relationships 
are substituted into Equation SB.2., and the time average of the product 
x(t- u)x(t+ t- v) is identified as the autocorrelation function of the in- 
put with argument (t-v +u), the autocorrelation of the output can be 
written in terms of the system impulse response and the autocorrelation of 
the input as (Carlson, 1968, p. 80): 

R (t) = i^i^h(u)h(v)R (t - v+u)dvdu (SB.S) 

y -CO .05 X 

Equation SB.S is an important intermediate result because it expresses the 
relationship between the input and output autocorrelation functions. It 
can be thought of as the autocorrelation domain equivalent of the time 
domain convolution integral of Equation SB.l. 

To obtain the power spectral density function for both the input and 
output processes, we take the Fourier transform of both sides of Equation 
SB.S. The resulting triple integral can be separated by rearranging the 
exponential of the transform to give an argument equal to the argument of 
the input autocorrelation function (Carlson, 1968, p. 81). This gives: 

•Wr (t) 5 =£3h(u)e + ^“* i du f h(v)e' )w dv J" » (t - v +u)e' i “ ,<T ‘ V + u) dT 

y ~ (SB. 6) 

The last integral is the Fourier transform of the input while the first two 
are the Fourier and inverse Fourier transforms of the impulse response. 
These transforms can be written in abbreviated form as: 

^tR y (T)} =_^ x " 1 {h(u)}^Ih(v)}j?tR x (T)} (SB. 7) 

Using the W iener-K hinchine relations and recognizing that the product of 
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the two transforms of h(t) is equal to the absolute value squared of the 
Fourier transform, we obtain the important relationship between input and 
output power spectra: 

s(w) = 1h(w)| 2 S (w) (5B.8) 

y x 

Equation 5B.8 is widely used in the analysis of linear systems and their 
response to random inputs (Thomas, 1969, p. 145). It clearly illustrates 
the effect of a linear filter on the spectrum of an input function. It is 
also used to determine filter shape for optimum design and spectral 
shaping. This relationship is also used in prewhitening and in the design 
of a linear predictive filter as needed in maximum entropy spectral 
analysis. 


C. Digital Convolution 

Digital convolution can be performed with two discrete functions by 
adapting the convolution integral for causal functions as given in Equa- 
tion SA.7. Since digital operations involve finite input data sets and 
digital filters of finite length, the convolution and filtering operations 
can always be thought of as involving causal functions. Approximating the 
continuous convolution integral with a Riemann sum gives 

Yt = 4t io h t-k\ (5ej) 

where: 

t = the output or observation time index. 

At =the sampling interval. 
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The correspondence between discrete and continuous variables is given 
below: 


yet) : y(tAt) =Yt 

h(t-T) : h(tit-k^T) =h t _ k (SC. 2) 

x(t) : x(kA-r) = ^ 

An example for a digital filter of length five and a time series of length 
six will now be given. Figure SC-I illustrates a graphical interpretation 
of digital convolution. The outputs for the example are: 

Yo = At(h 0 x Q ) 
y | = At(h^x Q + hOX| ) 

Y 2 = At(h 2 x 0 + hlxl +h^<2) 

Y3 = At(h 3 x 0 + h2X| + h I x 2 + h^ 3 ) 

Y4 = A t ( h 4 x 0 + h 3 x l + h 2 x 2 + h l x 3 + h d < 4^ 

Y5 = AtCh^x^ + 113X2 + h 2 x 3 + h| X4 + hOxg) 

Y6 = At(h 4 x 2 + h3X 3 + h 2 x 4 + hlx s + h ( j( 6 ) 

Yy = At(h 4 x 3 + h3X 4 + h2X s + h 1 xg) 

Y 8 =At(h 4 x 4 + h 3 x s + h 2 x 6 ) 

Yg — At(h 4 x 3 + h^Xg) 

Y I0 =it(h 4 x 6 ) 

These outputs represent the numerical "computation of the digital outputs 
for observation times starting at zero and ending at lOAt. The outputs YO 
through Y^ represent the transient response of the filter to the input, Y4 
through Y6 represent the steady-state output, and Y7 through YIO represent 
the "ringing down" of the filter. 
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Digital convolution or filtering can also be thought of as the 
weighted superposition of past values of x(r). As time goes forward, the 
past values are represented by smaller values of t. If this weighting is 
represented by coefficients a Q ' a | , a 2 , ••• the convolution can be 
written as: 


Yt 



(5C.3) 


The digital convolution concept does not depend upon having a filter 
with as many coefficients, a^, as there are data values, x i * As a general 
rule, the filter is of length M and the time series is of length N. It is 
desirable to have N much larger than M so that filter transient response 
will be minimized. 

The most commonly used formulation for digital convolution is given 
by Equation 5C.4: 


Yt 



(5C.4) 


The ranges on the indices are 


t =0, 1, 2, 3, 4, ••• (N + M ) 
k =0, 1, 2,3,4, ••• M 

and the values for x with indices less than zero are assumed to be zero 
(causality). The response of the filter and the resulting output can be 
separated into three distinct observation time intervals: 
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1. Yt for Ost £ (M - 1) is the transient response of the filter near 
the starting time. 

2 . Yt for Mst £l\l is the steady-state response. 

3. Y t for (N + l) ^t^(N+M) is the "ringing down" or decay response 
after removal of the input. 

Digital filtering and convolution can also be formulated with matrices. 
This procedure is straight forward if one uses Equation 5C.4. 

D. Digital Deconvolution 

A general method for solving the convolution integral and obtaining 
a unique solution does not exist. In special cases, the convolution 
integral can be solved but the required boundary conditions and class of 
functions are such that these special circumstances do not apply to most 
practical cases. For discrete data and digital filters, deconvolution is 
really a matter of definition and is an "unraveling" of the weighting 
applied by the filter coefficients. For most cases, digital deconvolution 
can be performed using the following method. 

Digital convolution was represented by Equation 5C.I. If the y's are 
measured and the h's are known, it is possible to use the set of (N+M +1) 
equations defined by (5C.I) to solve for the x's. This can be done re- 
cursively by solving the following system of equations: 

1 1 N-l 

*n ‘ AH Yn - a 0 kk a n- kx * <5DJ| 

These recursive equations can also be expressed exclusively in terms 
of the y's. When this is done, the resulting set of equations can be 
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thought of as a deconvolution filter with weights, b. . The "deconvolution" 
using this special filter is obtained from: 


x =!t i b . Y. 
n At k33 n-k k 


(5D.2) 


The bIS or fi I ter weights are obtained from the als of the original filter 
by using the following recursive formulas: 


b 


0 


1 


a 


0 



(5D.3) 


b 

n 


3 n 1 n '' 

a Q b 0 a Q k=l a n-k b k 


( n=2,3, ••• M) 



33 


VI. SAMPLING THEORY AND SAMPLED DATA FUNCTIONS 

In the usual sense, sampling theory is applied to sampled data systems 
in communication theory. In this application the user must consider such 
aspects as digital data transmission and reconstruction. The parameters of 
the system are usually under the control of the designer and can be opti- 
mized to suit a particular need. The signal is usually sampled at a rate 
well above the Nyquist rate and is sampled long enough to give any desired 
accuracy for the reconstruction. 

Sampling theory as applied to spectral analysis must be approached 
from a somewhat different viewpoint. First of all, the parameters of the 
transmitting system are not under the control of the experimenter. The 
"received signal" is usually a data function generated by some physical 
process that is being investigated. Since the nature of the received sig- 
nal is being investigated for its unknown spectral properties, it is dif- 
ficult to determine what sampling rate is necessary. Also, the amount of 
data needed to resolve an unknown spectrum cannot be determined in advance. 
The experimenter must solve this problem in an iterative manner by making 
reasonable assumptions about the spectrum to be measured. The sampling 
technique can then be studied to see if these assumptions are valid. A 
widely used technique for spectral analysis of an unknown spectrum involves 
the use of prefiltering to limit the spectral input to the sampler. This 
method is most successful if the sampling system can be adjusted to produce 
a range of filtered spectrums and sampling rates. As can be seen from the 
theory that will be discussed, the prefiltering method will produce an ac- 
curate estimate of the input spectrum that is within the filter passband. 
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A time function or time series generated by a real physical process 
can be generally represented in the frequency domain by a spectrum con- 
taining both continuous and line components. Because of the "uncertainty 
principle" between time and frequency, this function is theoretically 
limited in spectral bandwidth by the physical process of measurement. 
Drawing on an analogy from quantum mechanics, the process of observation 
will introduce a perturbation which limits the observed spectral bandwidth. 
Even if the process has an infinite bandwidth, any measurement of a data 
function would involve the convolution of that function with a "filter" 
inside the measuring system. 

Because of this effect, -every data function or set of empirical data 
will be bandlimited. To assure the user that the signal input to the 
sampler is bandlimited, it is wise to use the method of prefiltering. 

The method of data sampling can have a profound effect on an empirically 
derived power spectrum. The conditions of sampling such as sampling rate 
and number of samples are important parameters to be analyzed. The sam- 
pling technique is such a fundamental part of the measurement of a power 
spectrum that it is difficult to separate its effects from the actual 
power spectrum estimation technique. 

The basic concepts of sampling theory as applied to spectral analysis 
will be discussed in this chapter. It will not be possible to discuss all 
aspects of this broad topic but the most important ideas will be covered. 
Specific applications of sampling theory to the various aspects of spectral 
estimation will be presented in appropriate later chapters. 
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A. Sampling Theory - Basic Definitions 

The important parameters of a sampling system are the spectral band- 
width of the data function to be sampled, the sampling rate, the pulse 
shape of the sampling function, the output filtering for the sampler, and 
the amount of data measured. Figure 6B-I shows a block diagram of a sam- 
pling system that will be used for the system analysis of the sampler. It 
is assumed that prefiltering has already been applied to the data function 
and that post-sampling reconstruction or filtering will be applied at a 
later stage. For purposes of discussion it will be assumed that the proc- 
ess being examined produces a continuous time function. 

The time function to be measured, x(t), is called the sampled function. 
It is assumed to be continuous; bandlimited, and defined for all tine. 

The spectral bandwidth of x(t) will be assumed to be limited to 2w b * The 
output of the sampler is denoted by y(t) and is obtained by multiplying the 
sampled function, x(t), by the sampling function, m(t). m(t) is charac- 
terized by a repetitive pulse shape, p(t), and is a periodic function that 
can be represented by a Fourier series. The sampling system can also be 
modeled by the convolution of a Dirac comb function and a single pulse, 
p(t), which represents the basic pulse shape of the sampler. The system is 
modeled with system blocks such as multiplication and convolution so that 
the system analysis techniques given in Chapter V can be used in the de- 
rivations. 

The important relationship between the sampling rate (or frequency) 
and the Nyquist sampling rate (or frequency) can be discussed with the help 
of Figure 6A-I. The Nyquist frequency depends upon the spectral content of 
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Y(2rrf) 



a) Sampling frequency greater than the Nyquist frequency. 


Y(2nf) 



b) Sampling frequency equal to the Nyquist frequency. 


Y(2TTf) 



c) Sampling at less than the Nyquist frequency (undersampling). 


Figure 6A-1. Output Spectra for an Ideal Sampler Showing the Effects of 
Sampling Rate and Aliasing 
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the signal being sampled and is defined to be twice the highest frequency 

for which a nonzero spectral component of x(t) exists (Schwartz, 1970, 

p. 119). For the present discussion, the spectral components in x(t) will 

be limited to frequencies less than f^ so the Nyquist frequency is f N =2f^* 

The time interval between samples, , is called the Nyquist sampling 

z 'b 

interval. 

The sampler is characterized by the sampling frequency, sampling in- 
terval, and folding frequency. The time interval between samples (sampling 
interval) is T g and the sampling frequency is defined as the reciprocal of 

the sampling interval, f =^-. In this dissertation, the folding frequency 

s 1 

will be defined to be one-half of the sampling frequency, ff = 2 ^ 5 * The 
folding frequency is a characteristic of the sampler and is the highest 
frequency that can be sampled without aliasing effects. This definition 
and concept is important to the discussion of the theory of sampling. 

Other authors appear to define it in a different manner (Blackman and Tukey, 
1958, p. 32). 

If a signal is sampled at exactly the Nyquist rate, the Nyquist fre- 
quency and the sampling frequency will be equal and the folding frequency 
and the frequency of highest spectral content will be equal. This condi- 
tion is often used as an example in discussions of sampling but does not 
illustrate the important differences between the characteristics of the 
sampled signal and the characteristics of the sampling device. 

The sampling rate can have a drastic effect on the spectral output of 
the sampler. The sampling rate must be greater than the Nyquist rate 
or serious spectral distortion will occur. From the viewpoint of the 



38 


sampling device, this is equivalent to saying that the input signal cannot 
have any spectral components above the folding frequency or aliasing will 
occur. This phenomenon, which is called aliasing, is illustrated in 
Figure 6A-I. Ths input spectrum to the sampler is assumed to be triangu- 
lar. Figure 6A - 1 a shows the spectrum of the output, y(t), when the sam- 
pling frequency is greater than the Nyquist frequency of the input. Figure 
6A - 1 b shows the output spectrum when the sampling frequency is equal to the 
Nyquist frequency and Figure 6A - 1 c shows the aliasing effect when the sam- 
pling frequency is less than the Nyquist frequency. Sampling at a rate 
that is less than the Nyquist sampling rate causes the output spectra to 
overlap. This overlap or "folding back" of the spectra creates erroneous 
spectral components or "aliasing" of the true spectrum. It is important to 
note that not all of the spectral information is rendered useless by alias- 
ing. The spectral components below a frequency of (f g - f^) remain unaf- 
fected. 

B. Sampling Theory - The System Equations 

Figures 6B-2 and 6B-3 represent the sampling system in both the time 
and frequency domains. Both representations are used because both provide 
useful system information. The solution for the output spectrum of the 
sampler will be obtained by solving the convolution interals by means of 
the F ourier transform. 

The sampling function, m(t), can be represented as the convolution of 
the pulse shape of the sampler with a Dirac comb function (See Equation 
6C.2): 

m(t) =§ p(T)Comb(t- T)dT (6B.I) 



y(t) 


x(t) =Sampled 
| Signal 


multi pi ication 





Comb (t) 



m(t) 



Figure 6B-1. Time Domain System Diagram for a Multiplying Sampler 
Showing Signal Waveforms 




Figure 6B-2. Frequency Domain System Diagram for a Multiplying Sampler 
Showing Amplitude Spectra 



41 


The sampling function is even and periodic so it can be represented by an 
even Fourier series 

met) = - C + E.C cos t) = £ pet - nT ) (6B.2) 

2 o n=l n V T S s' 

where CJ2 is the average value and T g is the sampling rate. 

The output of the sampler is obtained by multiplying the sampling 
function, met), and the sampled function, x(t). The resulting output can 
be" w ritten i n two ways: 

+CD 

yet) =x(t)m(t) =x(t) £ pet - nT ) (6B.3) 

n=-co s 

In the frequency domain, the convolution integral representing met) is re- 
placed by multiplication of the Fourier transforms of pet) and Comb(t) 

M(u£> = P(w)Comb(w) (6B.4) 


where: 

M(ui) =3* (®(0 } 

pew) 

Comb( w) = ^lcomb(t) } 


Also in the frequency domain, the multiplication of x(t) and met) is 
replaced by the convolution of their respective Fourier transforms: 


1 

yew) =ZE OX(X)M(ou- \)d\ 

-CD 


(6B.S) 


These system equations describe the operation of the sampler and will now 
be used to derive equations for the output frequency spectrum and the re- 
constructed signal. 
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C. Output Frequency Spectrum 

The output frequency spectrum of the sampler can be obtained by 
making use of the special properties of the Dirac comb function or by 
expressing the sampling function as a Fourier series. Each method will 
now be briefly presented. 

The reason for expressing the periodic sampling function, met), as the 
convolution of a single pulse, pet), and a Dirac comb function is that the 
spectrum of met) can be expressed as the product of the spectrum of pet) 
and the spectrum of Comb(t). Since the spectrum of a Dirac comb function 
is another Dirac comb function, the convolution integral of Equation 6B.5 
is easily solved by using the sifting property of delta functions. First, 
the amplitude spectral density function for met) is: 

M(W) = P(a))Comb(ui> (6C.I) 

The Dirac comb function is written in the time domain as: 

+=> 

Comb(t) = _S 6(t - nT ) (6C.2) 

n — —co s 

The Fourier transform of the Dirac comb function is itself a Dirac comb 
function (Reference Data for Radio Engineers, 1968, p. 42-2): 

3^" (Comb(t)} = Comb(ui) = ~ tfsCLu-nJ^ (6C.3) 

s n=-® 1 s 

Since the output spectral function is obtained from the convolution 
of M(w) and X(w), we can write yew) as 

1 

Y(ui> = ^ jX(l)P(u)* X)Comb(u)- \)d\ (6C.4) 
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where M (w- X) =P(w- X)Comb(uj- X) . Explicitly writing out the series for 
the Dl'.rac c omb function and rep^Cl.ng wxth w 1 th e convolution -l.ntegral 
of Equation 6C.4 becomes: 

■ fr co j 

Y(uj) = 2~ J* [x(X)p(iu- \)uu s 6(uu- X - na^)]dX (6C.S) 

Equation 6C.5 is reduced by cancelling common factors and inter- 
changing summation and integration to give: 

1 +00 |+® 

Y(w) =70 2 I X(X)P(uo- X)6(co- X - nw )dX (6C.6) 

i s n=-=a i- s 

The integral can be eliminated by making use of the sifting property 
(Appendix II, Part E) of the Dirac delta function. Using the sifting 
property for Equation 6C.6, the convolution integral becomes 

+OD 

J X(X)P(u>- X)6(X - w+nw s )dX = X(w- nw s )P(nw s ) (6C.7) 


and the output spectrum for the sampler can be written as: 

Y(w) =-T E°P(nw )X (w- nw ) (6C.8) 

s n=-co s s 

The term P(n^) represents the Fourier transform of the sampling pulse, 
p(t), as it is periodically sampled at values nw s in the frequency domain. 
Since this amplitude function is a constant with respect to the frequency 
variable, m, and is only dependent on the sampling frequency w , it can be 
replaced by a real coefficient defined by: 


p n = *<™ s > 

s 


(6C.9) 


The output spectrum can now be represented by the infinite series 
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4co 

Y(uu) = Z p X(ui-niu ) (6C.I0) 

n=-» n s 

with constant coefficients and a periodic repetition of the amplitude 
spectrum of the sampled data function, x(t). Before interpreting the 
meaning of the output spectrum, the alternate derivation using the Fourier 
series representation for the sampling function, m(t), will be given. 

The sampling function is represented by the Fourier series of Equation 
6B.2. The Fourier transform of this series given by 

CO 

M (w) = TTC. 6 ( (I)) + E.C tt [6(nuj -w)+6(nuj+w)] (6C.II) 

0 n=l n s s 

is used in the convolution integral for the output spectrum given by Equa- 
tion 6B.5. The Fourier transform for m(t) is substituted into (6B.5) and 
integration and summation are interchanged to give the integrals: 

1 

y(w) =j Co CX(X)6(X- (n)dX 

+ -r E.C r X(X) C6(X - w+nw ) + 6(X- w- nw ) ]d\ (6C .12) 

2 n=l n _- oo s s 

The sifting property of the Dirac delta functions is used to eliminate the 
integrals and give an output spectrum of: 

1 1 1 “ 

Y((d) =4 C X(uu) + Z C X(u)-nm ) + ■£ Z C X(uj- nm ) (6C.13) 

£ o xi s 4 n — i n s 

n=-® n ” 1 

The first term represents the original spectral function centered about 
zero frequency and amplitude scaled by a factor of C q / 2. The second term 
is the series of all the negative frequency spectral terms and the third 
series represents all of the positive spectral terms. If these series are 
identified term by term with the previous result given by Equation 6C.I0 we 
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see that the coefficients are related as: 


P„ = 


P„ “ 


C / 2 

o 


C 12 
n 


C n =t p(n ^> 
s 


( 6 C .14) 


The function P(nv^) is even so the coefficients, p n or C p , form an even 
array (P n =P. n °rC n =C n )- 

A simple example will now be given to illustrate the application of 
the derived equations. Consider a sampler whose sampling function consists 
of a pulse train of narrow rectangular pulses as illustrated by Figure 60-1. 
The amplitude spectrum of x(t) will assume the arbitrary but oandlimited 
form shown in the figure. The amplitude spectrum of a single pulse of m(t) 
is given by the Fourier transform of a narrow rectangular pulse: 

*T S 

= "AeJ ^dt = - si n(-S uj) (6C.15) 


7 1 2 i- Mt 2A AT 

( p(t) } = J z AeJ °*dt = - sin(-S ui> 
' _*s 


w 


This equation can be written with a sine function: 

A T_ 


P(m) = A. AT r 


sin(- 


U)) 


AT, 


tu 


(6C.16) 


The pulse height of the sampler, A, is called the gain of the sampler and 
AT s is the pulse width. The product AAT g is the area under the rectangular 
pulse. For purposes of analysis, the pulse area is often set equal to 
unity. The spectrum of p(t), sampled at intervals corresponding to nu> , 


is used to obtain the coefficients, Pn- These coefficients are calculated 
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from: 

P(noj i ) = A A T 

S w 

The output spectrum for the rectangular sampler can be calculated from 
Equation 6C.I0 and the corresponding amplitude coefficients from (6C.17) 
and (6C.9). The output spectrum for the rectangular pulse example is shown 
in F igure 6C-2. 

As the pulse width of the rectangular pulse is made narrower, and the 
pulse height increased so that the pulse area remains constant, the rec- 
tangle function approaches a delta function. If the sampling function, 
met), is a Dirac comb function, the amplitude spectrum of a single delta 
function is a constant and the output spectrum simply becomes 

AT +» 

yew) =A s E X(w-nw) (6C.I8) 

T~ s n=—x> s 

because all the p coefficients are equal. The original spectrum is repro- 

" * T s 

duced periodically with an amplitude scaling factor of A - — . This type of 

s 

sampler is called an ideal sampler. The sampler that directly multiplies 
the incoming signal by a rectangular pulse shape is called a "natural" 
sampler, an exact-scanning sampler, or a multiplying sampler. When the 
data function or input signal is sampled with a sample-and-hold circuit, it 
is referred to as square-topped sampling. A similar analysis of the square- 
topped sampler yields an output spectrum which is given by: 

1 +* 

Y(uj) — - P(ui) E X(w- nw) 

T g Q.= —CD S 


AT 


sin nu) s ) 

~& s ” 
L T nuj s _ 


(6C.I7) 


(6C.19) 
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This spectrum differs from that of the exact-scanning spectrum in that the 
Fourier transform of the sampling pulse is now an explicit function of the 
frequency, id. This can be interpreted as a weighting function on the 
periodic spectrum that tends to lowpass filter the spectrum that would 
be produced by an ideal sampler. If m(t) is a rectangle function, the 
weighting function will be a sine function. This effect mayor may not be 
desired and certainly must be taken into account if a sample-and-hold 
circuit is used. 

From the general theory of sampling that has been presented it is 
possible to summarize some of the important features of a sampling system. 
The ideal or Dirac comb type of sampler produces a periodic output spectrum 
which has a fundamental shape corresponding to the spectrum of the input 
signal. This type of output is illustrated in Figure 6A-I. The effect of 
a finite linear sampling time in a multiplying sampler is to introduce a 
scale factor of P(nv^) into the periodic spectrum of X (w- nuj s ). This 
factor is not a function of frequency but its effect is to scale the ampli- 
tude of each term in the series. The shape of the pulse in this type of 
sampler does not affect the spectral shape of the baseband spectrum or any 
of the higher order spectra in the series. It does, however, affect the 
amplitude factor for each term. The sample-and-hold type of sampler, on 
the other hand, actually changes the spectral shape of the sampled input 
spectrum. This effect is easily accounted for by determining the Fourier 
transform of p(t). The output spectrum of a sampler is not a physical ob- 
servable and is usually processed by lowpass filtering to reproduce the 
original input signal. This filtering process is called reconstruction. 
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D. Data Function Reconstruction 

The input data function can be reconstructed at the output of the 
sampler by lowpass filtering the output spectrum. This is equivalent to 
convolving the output time function, yet), with the impulse response, h(t) 
of the lowpass filter. The reconstructed signal is represented by an esti- 
mate of the original input, x(t). The reconstructed output is given by the 
convolution integral: 

-H® 

x(t) = J h(T)y(t - r)dT (6D.I) 

-CO 

In the frequency domain, this filtering is represented by the multipli- 
cation of the output spectrum with the amplitude frequency response of the 
f i Iter (Equation 6e.S): 


(6D.2) 

s 

The convolution integral can also be written in tents of the original 
input signal and the sampling function as: 

. +® 

x(t) = J h(x)x(t- x)m(t- T)dT (6D.3) 

— 03 

The solution for the output estimate, x(t), is now obtained by solving 
these system equations. 

The convolution integral of Equation 6D.3 can be solved for the ideal 
sampler by replacing met) with a Dirac comb function and using the sifting 
property of the Dirac delta function as was done in Equation 6C.7. For the 
ideal sampler, the estimated output in the time domain becomes: 
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x(t) = E x(nT)h(t-nT ) (6D.4) 

n=-® S S 

This equation has an interesting mathematical form that is worth discussing 
because it gives insight into the sampling and reconstruction process. 

This series for the estimated function, x(t), uses a vector basis set which 
is derived from the impulse response of the lowpass filter. The amplitude 
coefficients are simply the data samples at times, nT s * It is significant 
to realize that any bandlimited signal, x(t), can be represented by an in- 
finite series with amplitude coefficients corresponding to the samples of 
x(t) at times nT g and basis vectors corresponding to the impulse response 
of an arbitrary lowpass filter. The only requirement on the reconstruction 
filter is that it completely removes the higher order spectral components 
of y(t). This requirement is satisfied if the transfer function for h(t) 
is zero for frequencies above u^. 

A commonly used example for a lowpass filter that is frequently seen 
in textbooks is that of the ideal rectangular lowpass filter. The impulse 
response of this type of filter is obtained by taking the Fourier trans- 
form. of a rectangle function with height n q and "double-sided" spectral 
bandwidth 2u^. This transform, is the well-known sine function and can be 
written as: 


h ( t) 


No 

2TT 2lI b 


sxn 




J 


L V 


(6D.5) 


This impulse response is used in the series of Equation 6D.4 to give the 
well known result 


X(t) 


+03 

2N f, 2 x(nT ) 
o d n=~°° s 


sin 2rr f b (t - nT g ) 
2TTf b (t-nT s ) 


(6D.6) 
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where is the "single-sided" spectral bandwidth, in HZ, of the lowpass 
filter. 

In most cases, the results obtained for the ideal sampler can be di- 
rectly applied to "exact-scanning" and "flat-topped" samplers. Usually 
only the amplitude factor is affected. For exact-scanning, the estimated 
output spectrum is obtained by lowpass filtering the spectrum given by 
Equation 6D.2: 

#(w) = P(0)H(w)X(w) (6D.7) 

The inverse Fourier transform is taken to give the output in the time do- 
main. The resulting equation will be the convolution of the impulse re- 
sponse of the filter and the input signal. The final result is given by 
Equation 6D.4 and the addition of a scaling factor to account for the 
finite gain of the sampler: 

x ( t) =2£2I x(nT ) h ( t - nT ) (6D.8) 

T<. n=— «° s s 

P(O) is the Fourier transform of the pulse shape of the sampling function 
evaluated at zero frequency. For the previous example of a rectangle 
function, this factor is, P(O) = A.flT • 

The flat-topped sampler introduces an additional complicating factor 
because the output power spectrum after lowpass filtering still contains 
the weighting function P(uu). This result is shown by filtering the spec- 
trum given by Equation 6C.19 with a lowpass filter to give the estimated 
output spectrum as: 

X ( uo) =H(w)Y(u)) =^r H( w)P( uj)X( oj) 


(6D.9) 
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The solution in the time domain involves two convolution integrals and can- 
not be solved in general. If the width of the sampling pulse is suffi- 
ciently small so that P(w) is essentially a constant over the frequency 
interval 0 to u^, the output in the time domain will be the same as that 
given by Equation 6D.8. This approximation is valid for most practical 
samplers. 

It is of practical interest to compute the spectral output of the low- 
pass filter by taking the Fourier transform of the estimated output given 
by Equation 6D.8. Since the only time dependent term in the equation is 
the time shifted impulse response, the Fourier transform is simply the in- 
finite series composed of the Fourier transform of h(t) and the shifting 
theorem phase factor: 

x(w) =^H(uj) 2 ° x(nT )e" jnuff s M ^ (6D.I0) 

s n=-= 

The infinite series part of (6D.IO) is exactly the form of a digital 
Fourier series with infinite sampling. This important result shows how 
the continuous spectrum given by Equation 6D.7 can be obtained from the 
lowpass filter spectrum and the discrete samples of x(t). Both of these 
quantities are known in a practical sampling system. 

If the lowpass fiiter has a rectangular passband, its amplitude spec- 
tral function, H ( w ) , will be a constant over the spectrum of x(t) and the 
output spectrum, $(w), can be computed, within a scale factor, by a digital 
Fourier series with infinite sampling. 
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E. Sampling Theory - Finite Record Lengths 

In practical sampling situations, the length of the input data 
function is either limited in time or else the data can be observed for 
only a finite time. This situation can be adequately modeled by using a 
sampling function which exists only over a finite time interval. For a 
total of 2N + 1 observations of the input signal, the sampling function is 
defined as: 

if 3 +N 

m(t) = Op(,-) I 6(t - nT )d,- (6E.I) 

»cd n— N s 

or in terms of the periodic pulse train as: 

4N 

m(t) = E p(t - nT ) (6E .2) 

n— N s 

The sampling function is periodic only over the interval of the defined 
pulses. In effect, the sampling function has been multiplied by a rec- 
tangular window function of width (2N + I)T. This forces the amplitude 
spectrum of m(t) to be infinite and continuous. This effect does not 
cause a problem with the analysis of the system but does force the ob- 
server to accept, at best, only an estimate of the "true" input spectrum. 

The amplitude spectrum of the sampling function can be obtained by 
taking the Fourier transform of the finite pulse train of Equation 6E.2. 
The transform is simply the transform of the pulse, p(t), with a phase 
factor determined by the shifting theorem. This result may be written as: 

M(W) = P(uu) 1? e~ jnuirs (6E.3) 

n=-N 

If m(t) is modeled by an infinite pulse train multiplied by a window 
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function, the resulting amplitude spectrum is obtained by convolving the 
amplitude spectrum of the window function with the amplitude spectrum of 
the infinite series. The infinite series can be written as a Fourier series 
so its Fourier transform has already been given by Equation 6C.II. The 
Fourier transform of a window function of unity amplitude and width 
(2N + I)T is the same as that for a rectangular pulse. This transform is 
given in Equation 6C.I6 where A = land &T S = (2N + I )T s ' The convolution 
integral can be solved by using the sifting property of the Dirac delta 
function to give: 


/ 2N + 1 +« 

M( w) = ( )T E C 
2 s n=-» ’ n 


,2N+ . 

sm(— 2“ |) TS ( W - 




(6E.4) 


The amplitude coefficients, C , are determined from the amplitude spectrum 
of the pulse shape as given in Equation 6C.I4. When this substitution is 
made into (6E.4), the desired alternate form for the spectrum of the mul- 
tiplying function is obtained as: 


M (W) = (2N+i; 


n=-» 


-Ho slone2 l\g+l )T s(w-nW„ 

2 P(nw) f 2N+I 

s L ( _ )T (w- nw ) 


J 


(6E.5) 


Equations 6E.3 and 6E.5 represent two equivalent series representations 
for the amplitude spectrum of m(t). Both of these representations have 
their usefulness in certain applications and insight into the properties 
of the sampling function can be obtained from both. The sampler output 
spectrum will now be computed using the representation of (6E.3). The out- 
put spectrum of the sampler, before filtering, is a series of 2N + 1 func- 
tions obtained by convolving the spectrum of x(t) with the spectrum of pet) 
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and a multiplicative phase factor: 

+N +CD 

Y(w) = s i- J* XWPCau-^e'^^^SdX 
n=-N 2tt tec 


(6E.6) 


If the sampler is ideal, the Fourier transform of the sampling pulse 
will be a constant and the convolution integral for the output spectrum 
of (6E.6) will be reduced to: 

Y(ut)=~ S N e" jimT s f X(\)e 4jXnT sdX (6E.7) 

2tt n=-N 

The integral in (6E.7) is the inverse Fourier transform of the amplitude 
spectrum of x(t). If we identify the inverse domain variable as nT s , the 
output spectrum can be simplified to the finite series: 


_ +N _ , (jjyf 

Y(U)) — E x(nT )e s 
n=-N s 


(6E.8) 


In the frequency range of Iwi Y(w) is an estimate of the amplitude 
spectrum, X(w). As the number of samples of the input function becomes 
large, the estimate converges to the "true" value. This expression for 
Y (w) is identical to that of a discrete Fourier transform for a bandlimited 
spectrum. 

The output spectrum computed using Equation 6E.8 involves an infinite 
series of convolution integrals which convolve the sine function in M(w) 
with the "true" spectrum X(w). This convolution in the frequency domain 
tells us that the spectrum of X(w) is smoothed by the finite observation 
process. Spectral smoothing reduces the spectral resolution of the obser- 
vation and limits the ability of the measurement process to resolve two 
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closely spaced line spectra. The problem of spectral smoothing will be 
discussed in detail in several later chapters. 

Other discussions of sampling theory as applied to communication 
systems can be found in (Carlson, 1968, Ch. 7), (Thomas, 1969, Ch. 7), 
and (Schwartz, 1970, Ch. 3). 
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VII. THE FIRST PRINCIPLE OF DATA REDUCTION 

The reduction of real physical data often requires estimation schemes 
which include, either overtly or covertly, algorithms which modify the data. 
Often the selection of a particular estimation scheme results in observer 
bias being introduced into the data processing. A common example of this 
type of observer bias occurs in the selection of a Lag window as is done 
in the B lackman-Tukey method of spectral analysis. Merely by selecting 
different types of lag windows, the observer can compute different esti- 
mated power spectra from the same observed data. In theory, we say that 
the autocovariance function and the power spectral density function form 
a Fourier transform pair. When using a lag window, the inverse transform 
of the computed power spectrum will not "give back" the initial autoco- 
variance function. This type of processing is in violation of the First 
Principle of Data Reduction (Abies, 1974): 

"The result of any transformation imposed on the experi- 
mental data shall incorporate and be consistent with all 
relevant data and be maximally non-committal with regard 
to unavailable data." 

By the very nature of data processing and estimation it is ordained 
that the FIRST PRINCIPLE will be violated to some degree. The fact that 
all sampled data is finite in length causes this violation. Whenever 
real physical data is modeled by a mathematical function and a criterion 
of goodness is chosen, it follows that certain assumptions must be made 
about how the model fits outside the domain of measurement. 


The first example of this is the modeling of a finite time series 
with a Fourier series. The complex amplitudes are computed by assuming 
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that the length of the observed series is the Fourier period, that the 
observed time series is periodic outside the interval of observation, and 
that the sampling rate is more than the Nyquist rate. If the observer 
has not sampled the data in such a way that these assumptions are valid, 
the computed complex amplitudes may be poor estimates of the "true" complex 
amplitudes. Observer bias has entered into the scheme by the method of 
data taking and by the tacit assumptions involved in using the digital 
Fourier transform. The irony of this situation is that the computed com- 
plex amplitudes can be used to generate the observed time series exactly. 

Of course we must realize that an infinite number of Fourier series may be 
used to describe a finite time series but only one was computed. 

A second example of time series modeling is the autoregressive-moving 
average time series. I n maximum entropy spectral analysis, a least-square 
error criterion is applied to fit the observed time series. The assumptions 
in this method are that the process can be represented by an n^-order 
autoregressive-moving average time series and that the observation time 
was long enough to permit an exact calculation of the autoregressive co- 
efficients. Also, it is assumed that the infinite time series can be 
computed from the estimated series. This is analogous to the assumption 
of periodicity in Fourier analysis. 

Although the first principle can never be ideally achieved it is worth 
striving for and any proposed data processing scheme should try to minimize 
its effects. Techniques which try to minimize the effects of data windows 
are referred to as "data adaptive spectral analysis" (Lacoss, 1971). 
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VIII. DATA ADAPTIVE SPECTRAL ANALYSIS 

The study of spectral analysis using only hypothetical or ideal data 
functions provides little insight into the practical problems involved in 
the study and analysis of empirical data sets. The reason for this is 
that a strictly mathematical analysis of data functions and spectral func- 
tions depends on the existence of infinite domains and infinite amounts of 
data. Attempts to modify the theory of spectral functions to apply to 
finite empirical data sets have been plagued by many problems. The theory 
that applies to infinite domains must be greatly modified to handle finite 
data sets. Spectral analysis methods for empirical data sets involve the 
definition and selection of various estimation procedures that, hopefully, 
result in realistic estimates of the "true" spectrum. A study of the 
various estimation procedures for spectral analysis constitutes the major- 
ity of effort in this type of research. A brief discussion of commonly 
used spectral analysis estimates will now be given and are used as an il- 
lustration of why data adaptive procedures are being sought. 

In most applications of spectral analysis, the goals are to determine 
the existence of "peaks" in the frequency spectrum, the location of these 
peaks, and their spectral power. The reason that the peaks are sought is 
that they contain the interesting information needed for scientific anal- 
ysis. Using definitions from information theory, a spectrum which is flat 
would be called a maximum entropy spectrum while a spectrum containing 
only one amplitude value would contain zero information. 
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Often, the amplitude spectrum is estimated directly by taking the 
fast Fourier transform of the empirical data. This procedure produces a 
discrete amplitude spectrum which is based on the assumptions that the 
data is periodic and spectrally limited both in the number of spectral 
components and the frequency of the highest component (See Chapter XII). 

In statistical studies of spectral functions derived from sample functions 
of a real process, the amplitude spectrum is avoided for theoretical 
reasons because it may not exist or it will not give a power spectrum es- 
timate that converges. For this reason an alternate definition is often 
chosen. 

This alternate definition is the famous W iener-K hinechine relations 
(Appendix I, Part F) using the autocorrelation function. This definition 
is preferred because of the previously given theoretical reasons. There 
are severe problems associated with the use of an autocorrelation function 
for empirical data. First, the estimated autocorrelation is badly trun- 
cated. If the spectrum is estimated by taking the Fourier transform of 
this truncated function, many additional frequency components are intro- 
duced that are not in the original spectrum. Second, this estimate is 
obtained from only one sample function of the process. For statistical 
reasons, many samples are needed to make the autocorrelation function con- 
verge to the "true" value. Third, the algorithm used to determine the es- 
timated function uses less data as the lag value increases. This causes 
the function to be a poor estimate for large values of lag. Practical 
schemes for estimating the power spectrum of a random process must take 
these undesirable effects into account and minimize their effect on the 
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accuracy of the estimate. 

The most commonly used method of spectral analysis of empirical data 
is the Blackman-Tukey method (Blackman and Tukey, 1958). This method uses 
the autocorrelation definition for the power spectral density of a random 
process. Their approach to the problems associated with the estimated 
autocorrelation function was to introduce a lag window function which 
would be multiplied with the estimated autocorrelation function. The lag 
window function was chosen so that it would weight the autocorrelation 
values in such a way that values near zero would be emphasized and large 
lag values would be removed or smoothed to zero. Their fundamental assump- 
tion was that the modified estimate of the autocorrelation function would 
give a respectable estimate of the smoothed values of power spectral den- 
sity even though the weighting process actually produced a poor estimate 
of the "true" autocorrelation function. This assumption has been surpris- 
ingly effective in a number of practical applications and accounts for the 
success of this method. 

Although the Blackman-Tukey method is probably the best method in 
common use, it has serious drawbacks in many scientific applications. 

First, for limited amounts of data, the estimated autocovariance function 
is severely truncated and the process of "windowing" further reduces the 
range of useful lag values. The method recommends that only about 5 to 10 
percent of the estimated autocovariance function be used. This procedure 
effectively removes much valuable data that might otherwise be used in the 
estimation of the spectrum. Second, the selection of a lag window intro- 
duces bias into the data and tempts the observer to "select" a window func- 
tion that produces the "most agreeable" spectrum. The use of a lag window 
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function is also found to be objectional on theoretical grounds because 
it causes "negative" power spectra to be produced. Third, in many practi- 
cal situations, the goal of spectral analysis is to detect and resolve two 
closely spaced spectral peaks but the B lackman-Tukey method of using a lag 
window produces a smoothed estimate of the "true" spectrum resulting in a 
loss of spectral resolution. The most obvious questions to be asked are: 

1) How can the data be processed to use all of it and still avoid the 
problem of truncation and 2) How can the smoothing effect associated with 
"windowing" be minimized to produce the best spectral resolution? Data 
adaptive spectral analysis methods are an attempt to answer these questions. 

The discussion of the drawbacks in the B lackman-Tukey method should 
not be viewed by the reader as an attempt to discourage the use of this 
method. On the contrary, there are many practical situations where this 
method gives very good results. It is important that the reader be aware 
of the problems attendant to the use of a lag window and the possible al- 
ternatives to its use. As is often the case, the user will probably try 
several methods and compare the results with his a priori knowledge about 
the measured process. 

There has been much work done on the selection of an "optimum" window 
function and many window functions are in common use (Otnes and Enochson, 
1972)* The primary goals of this work have been to retain as much spectral 
resolution as possible and still minimize the production of "negative" 
power spectra. One cannot escape the fact that the concept of a window 
function will always be valuable in the discussion and applications of 
power spectrum estimation techniques. Windowing can, however, be ap- 
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proached in entirely different ways if alternate modeling techniques are 
employed such as those used in data adaptive spectral analysis. 

In recent years, data adaptive methods of spectral analysis have re- 
ceived increasing attention from experimentalists. These methods attack 
the problem of limited data and are producing surprisingly good spectral 
estimates using data that cannot be easily processed by the Blackman-Tukey 
method. Data adaptive methods are based upon assumptions which allow 
limited degrees of smoothing and extrapolation to be applied to the esti- 
mated autocorrelation function. Within the limits of the various assump- 
tions, this technique essentially produces an infinite function which can 
be easily handled by conventional theory. This approach essentially cir- 
cumvents the need to select a window function but has not eliminated the 
theoretical requirement that a truncated autocorrelation function needs 
some type of "windowing". The way in which a truncated autocorrelation 
function may be smoothed and extrapolated is governed by the criterion that 
is selected for the particular estimation routine used. Data adaptive 
methods are so named because the empirical data actually governs the opti- 
mization criterion used for smoothing and extrapolation. It can be said 
that the problem of windowing adapts itself to fit the nature of the data. 

Lacoss (1971) published a widely referenced paper in which he discusses 
the mathematical formulation of such data adaptive techniques as maximum 
likelihood and maximum entropy. His use of a matrix formulation of the 
methods adds a valuable dimension to their understanding and provides the 
reader with more theoretical insight than can be obtained by studying the 
algorithms only. This paper is highly recommended reading because it com- 
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pares the data adaptive methods with more "conventional" methods. 

This discussion has been given to provide the reader with the necessary 
insight into the problems associated with the estimation of power spectra. 

It should be emphasized again that the problem of truncated data is not 
considered by conventional theory and that various estimation techniques 
must be used. These estimation techniques involve either the direct use 
of a windowing function as in the B lackman-Tukey method or smoothing and 
extrapolation as done in the data adaptive techniques. The selection of a 
particular technique of spectral analysis is discussed in Chapter IX. The 
implementation of some of these various techniques is discussed in later 
chapters. 
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IX. METHODS OF SPECTRAL ANALYSIS 

The selection and application of a particular method of spectral 
analysis is a very complicated task and offers many challenges for the 
potential user. The development of a method usually begins by selecting 
known estimators and then modifying them to fit the conditions of the prob- 
lem. This process of development often requires a unique mixture of theory, 
art, and "luck" before a suitable solution is obtained. 

Estimators obtained from the theory of spectral functions can be 
greatly improved if as much information as possible is incorporated into 
the development of a technique. Physical boundary conditions, a priori 
assumptions, and other information should be used. Modifying the estimator 
in these ways can be thought of as a type of model fitting. It should be 
apparent that effort expended in obtaining the best starting model will pay 
off in producing the best final results. 

It is ironic that the selection of a "best" spectral estimator often 
depends upon a priori assumpcions that mayor may not be valid for the 
empirical data to be analyzed. For example, the computation of an ampli- 
tude spectrum by Fourier analysis techniques depends upon the assumption 
that the function is periodic and that it has been observed for exactly 
one Fourier period. This assumption is seldom true but the amplitude 
s.pectrum is estimated this way by most users. Another example is the 
representation of empirical data by an autoregressive series. It must be 
assumed that the process produces a sample function that is an autoregres- 
sive series before the autoregressive spectral estimator can be used. 

Again this assumption is seldom true but the autoregressive spectral esti- 
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mator is used with good success and can give very good approximations to 
the spectrum of a periodic function provided the autoregressive series has 
enough coefficients. The validity of these various assumptions will affect 
the accuracy of the spectral estimate and offers a basis for judging the 
"goodness" of a particular technique. The effect of the various assumptions 
will be discussed in detail in later chapters. 

A data function is characterized in the frequency domain by its ampli- 
tude spectrum or its power spectrum. These spectrums are estimated by ap- 
plying various filtering techniques and using the theory of spectral func- 
tions given in Chapter IV. Since spectral estimation theory is based on 
the use of filtering, Fourier transforms, and autocorrelation functions, 
the development of suitable estimators is primarily concerned with obtaining 
good approximations to the integrals in these techniques. Estimators for 
the amplitude spectrum are based on the discrete Fourier series or trans- 
form. Estimators for the power spectral density function are based on 
filtering and/or the autocorrelation function and its transform. The auto- 
regressive spectral estimator is based on the discrete form of the auto- 
correlation companion function discussed in Chapter XV. The various es- 
timators can also be described in terms of linear predictive filtering and 
recursive digital filtering. 

A. Computing M ethods 

It is difficult to place the various spectral analysis methods into 
distinct categories because of the large number of variations that can be 
applied. For the benefit of the reader, a brief listing will be given in 
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an order that seems the most logical to the author. Other methods of 
presentation based on different viewpoints would be equally logical. The 
amplitude spectrum can be obtained by computing the Fourier transform of 
the data function. This scheme seems suitable and no other methods are 
commonly used. The computation of the power spectrum is quite a different 
matter. Most schemes are based on the autocorrelation function, recursive 
digital filtering, or the autoregressive spectral estimator. Methods which 
avoid the explicit use of a window function are called data adaptive methods 
(See Chapter VIII). Estimators for the power spectrum include: 

1. The B lackman-Tukey method (autocorrelation function, window func- 
tion). 

2. The direct Fourier transform method (amplitude spectrum squared). 

3. Recursive digital filtering. 

4. Data adaptive methods, 

a) maximum likelihood method. 

b) maximum entropy method, Burg algorithm. 

c) maximum entropy method, Levinson algorithm. 

5. B artl ett esti mator. 

6. A utoregressive spectral estimator. 

These various estimators do not represent completely independent methods. 

For example, the data adaptive methods and the autoregressive method can 
be formulated with the same mathematics that is used to describe recursive 
digital filtering. Likewise, the autocorrelation function and lag window 
function can be represented by analogous equations representing the digital 
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filter. The B lackman-Tukey method could then be formulated with the proper 
selection of a recursive filter. The selection of a particular representa- 
tion often depends on the viewpoint adopted by the user or his estimation 
of the most "economical" approach. 

B. Data Collection and Experimental Design 

The observation and collection of data can have an important inter- 
active effect with the chosen method of spectral estimation. The experi- 
menter should consider the effects of data collection on the errors intrin- 
sic to a particular spectral estimator. An observation system model for 
spectral analysis studies is shown in Figure 9B-I. A hypothetical sample 
function, s(t), is used to model a ergodic real process. It is assumed 
that s(t) will generally contain both random and periodic components and 
is properly well-behaved so that its autocorrelation function and power 
spectrum exist and are well-defined. In theory, the "true" sample function 
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Figure 9B-I. Observation System Model for Spectral 
A nalysis Studies 
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of a process is inaccessible to direct observation. This is because the 
observation process limits the. observation bandwidth and total number of 
data samples. This has the effect of limiting the spectral bandwidth and 
resolution of any spectral estimator. 

The inherent bandwidth limiting effects are modeled by a filter with 
impulse response, h(t). This filter may represent the time constants 
associated with probes, cables and other electrical conductors or it may 
represent the use of prefiltering for the sampler. Prefiltering is neces- 
sary for the successful operation of a practical data sampler. This filter 
is designed to have a sharp cutoff characteristic to limit the maximum 
spectral component that is transferred to the sampler. The filter should 
be designed so that it effectively attenuates all frequencies above the 

folding or aliasing frequency, fa = The filter passband and the 

s 

sampling rate can be properly designed to make the problem of aliasing 
negligible compared to spectral mixing errors in the estimator. 

The effects of finite data sampling are modeled by a multiplying or 
"natural" sampler. The effects of data sampling on the spectrum of an in- 
put function were discussed in Chapter VI. The observable function, d(t), 
at the output of the sampler will be called the data function. This will 
be the function that represents the actual data used by the spectral esti- 
mation routine. The data function represents all that is observed about 
the generating process and its sample function, s(t). All estimates of 
the spectrum of the process will depend on the observed data in the data 
function. 

The spectrum of d(t) can be considerably different than the spectrum 
of s(t) if proper experimental design is not used. First, the observation 
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filter will pass only those spectral components of s(t) which are in the 
filter passband and will attenuate or reject those components outside the 
passband. The filter output is the convolution of the sample function, 
s(t), and the filter impulse response, h(t). If the spectral estimator is 
capable of producing highly accurate estimates of the spectrum of d(t), it 
may be necessary for the experimenter to deconvolve or "inverse-filter" the 
output to obtain a better estimate of the spectrum of s(t). Second, the 
sampler will introduce frequency multiplication and some aliasing of the 
spectrum passed by the filter. Third, the limited number of available data 
samples will limit the spectral resolution of the estimator. All of these 
effects must be accounted for if acceptable estimates are to be obtained. 

Many of the problems caused by the observation and sampling of a 
process can be minimized by proper experimental design. For example: 

1. The filter bandwidth can be made wide enough to pass all signif- 
icant spectral components of s(t). 

2. The sampling rate can be made high enough to avoid aliasing 
problems. 

. 3. The output of the sampler can be digitally filtered so only the 

baseband spectrum (also called the principle alias) is determined. 

Even with careful experimental design, the estimated spectrum of a 
process can never converge to the "true" spectrum because of the intrinsic 
spectral estimation errors. A finite, sampled-data function will always 
be corrupted by aliasing, leakage, spectral smoothing, and statistical 
variabi I ity. 
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C. Data Preprocessing and Intrinsic Spectral Estimation Errors 

Empirical data that must be used for spectral studies often is cor- 
rupted with errors that are due to the observation technique or are in- 
herent to the estimation process. The user can minimize these effects by 
properly preprocessing the data and choosing an estimation method that will 
minimize intrinsic effects. The following data preprocessing techniques 
can be used, when needed, to improve the accuracy of a chosen spectral 
esti mator: 

1 . Wild poi nt editi ng. 

2. Trend removal. 

3. Remove average value ("de" value). 

4. Smoothing and decimation. 

5. Prewhitening. 

6. Rejection filtering. 

Wild point editing is necessary when the observation procedure is 
likely to introduce data values that are erroneous and not related to the 
"true" data. These types of errors are usually due to equipment malfunc- 
tions and gross errors comnitted by the observer. A "wild" data value can 
cause considerable trouble in an estimation routine, if the amount of data 
is limited. For instance, a least-squares fit can be greatly influenced by 
a data value that is far removed from the mean. The least-squares method 
would place great importance on the value because it has a small probability 
of occurrence. The selection of an editing scheme is largely a matter for 
user definition and depends on the type of data to be collected. A scheme 
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that rejects data points with large variance is discussed in Otnes and 
Enochson (1972, p. 56). 

Trend removal is an euphemism for treatment of nonstationary data 
that has a mean value that varies over the interval of observation. This 
violation of the assumption of stationarity is not serious if the rate of 
variation of the mean is very slow compared with the rate of fluctuation 
of the data. This slow variation in the mean value can be removed by 
model fitting with a low-order polynomial, partial sinewave, exponential 
function or other estimation schemes listed in Chapter III. Trend removal 
causes an irrevocable change in the low frequency spectrum of the data and 
cannot be used if low-frequency spectral estimation is important. Trend 
removal can be treated as a process of highpass filtering and appropriate 
filtering techniques can be applied. 

The removal of the average value from the data function is really a 
simple form of trend removal where the "de" spectral component is "filtered" 
out. The accuracy of most spectral estimators can be improved by removing 
the average value because this minimizes the "spectral mixing" effect 
(Chapter XI). This "spilling over" or "leakage" of one spectral component 
onto the frequency of another is an important source of estimation error 
and "de" removal is almost always recommended. 

Smoothing and decimation can be applied to data functions that are 
sampled at several times the Nyquist rate. As discussed in Chapter VI, 
oversampling 'increases the spectral bandwidth and the total number of data 
samples. If the total spectral bandwidth is not needed to display the 
desired spectrum, it is often desirable to filter or smooth the unwanted 
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spectrum. This will minimize the effects of spectral mixing and reduce any 
high frequency noise that may have been introduced in the measurement pro- 
cess. Decimation (Blackman and Tukey, 1958, p. 45) refers to the process of 
reducing the total number of data points by selecting only every second 
or third sample. This process has the effect of reducing the sampling 
rate and increasing the sampling interval to 2&T, 3&T, etc. Decimation 
is practical only when the data are highly oversampled and the desired 
spectral bandwidth is well-defined. Oversampling of data has some desir- 
able features such as improved estimation accuracy and minimization of 
aliasing problems that should not be overlooked. Decimation should only 
be applied when the reduction of the total number of data samples is ab- 
solutely necessary due to processing efficiency problems. 

One of the most frequent causes of errors in spectral analysis is 
that due to spectral mixing (called intermodulation distortion by Blackman 
and Tukey). This problem is especially severe when the data spectrum 
contains one or more peaks which have a high spectral power. Some of this 
power will "leak" into another part of the spectrum and give an erroneous 
contribution to the spectral amplitude. The leakage power can be so high 
as to cause the true spectrum to be completely "masked" by the leakage 
contribution. The spectral mixing problem is greatly reduced for a data 
spectrum which is essentially flat or "white". It is often observed that 
spectral estimators give the most accurate results for a "white noise" 
spectrum. Prewhiteni ng refers to the process of adding a synthetic signal 
to the measured data signal that will add to the data spectrum to make it 
essentially flat. This "prewhitened" signal is then spectrum analyzed 
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and the resulting spectrum corrected for the added spectrum. 

Prewhitening essentially corrupts the data with additional noise and 
may cause the "purist" to reject its use. This should not be the case, 
however, because it is designed to improve the esti mation and not the 
theory of spectral analysis. There is a strange aspect of prewhitening 
that needs further discussion. This involves the design of a filter or 
other processor that is used to prewhiten the data. One might suppose 
that the technique used to determine the required synthetic spectrum might 
also be suitable for estimating the data spectrwn directly. This is in- 
deed the situation and the data adaptive techniques that employ recursive 
digital filtering are used to compute a "prewhitening filter" that is used 
directly to estimate the data spectrum. The prewhitening filter concept 
is the basis for the autoregressive spectral estimator and the maximum 
entropy spectral estimator. 

Rejection filtering is yet another attempt at reducing the effects of 
spectral mixing. This preprocessing technique should be used if the data 
spectrum has an unusually strong peak. The amplitude of this peak can be 
reduced by notch filtering to make it comparable with the rest of the 
spectrum. The application of this technique usually requires at least 
one 1 1 ru n" of the data function through a spectral estimator that will 
detect this strong peak. The characteristics of the rejection filter 
must be retained in the processor so that the spectral peak may be "recon- 
stituted" in the final estimate. 

The preprocessing of empirical data and the careful selection of a 
spectral estimator are necessary for successful spectral analysis. These 
processing and selection techniques are designed to minimize the effects 
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of i ntri nsi c spectral esti mati on errors. I ntri nsi c errors are those 
which are present for all estimation methods. These errors often do not 
appear explicitly in the analysis of the estimation method and may cause 
unexpected problems if the user is not aware of them. The most common 
errors are listed as follows: 

1. Aliasing (undersampling). 

2. Spectral mixing (intermodulation distortion or "leakage"). 

3. Spectral smoothing. 

4. Statistical variability. 

5. Lack of stati onarity . 

A I i asi nq or frequency fold-over is caused by undersampling or sampling 
at a rate that is less than the. Nyquist rate. Aliasing will be present to 
some extent in all practical sampling systems. The undesirable effects of 
aliasing can be minimized by the use of sharp cutoff filters and adequate 
sampling rates. Every attempt should be made to make the effects of 
aliasing negligible compared to the effects of spectral mixing. Aliasing 
is discussed in detail in Chapter VI. 

S pectral mixing is the most severe problem in spectral analysis. 

Most estimators are designed to minimize its effects but it still causes 
problems when the data spectrum has one or more high spectral peaks. This 
problem can be described by a "leakage" of the power at one spectral 
frequency into another frequency. Lag window functions that are used in 
the B lackman-Tukey method are usually chosen to minimize leakage. The 
term leakage is derived from the analogy between a window function and the 
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field pattern of an antenna. The one-dimensional aperture distribution 
and the far-field power pattern for a beam antenna are reciprocal Fourier 
transforms as are the lag window function and the spectral window function. 
The minor lobes of the antenna cause "leakage" of the power that is wanted 
in the main beam. Likewise, the lag window function causes leakage of 
the spectral power at the observed frequency into adjacent frequencies. 

The problem of spectral mixing is present whenever the observed data func- 
tion is truncated. This is true for all known estimators. In most prac- 
tical schemes, the problem of leakage can be traded off against the prob- 
lem of spectral smoothing. 

Spectral smoothing is also caused by the truncation of the observed 
data function. The lack of enough data to resolve closely spaced spectral 
components causes two nearby peaks to be "smeared" together. The only 
solution to the smoothing problem is to observe the data function for a 
longer period. In the B lackman-Tukey method, spectral smoothing is an 
obvious result of the convolution of the spectral window function with the 
"true" spectrum. As the period of observation increases, the spectral 
window function becomes very narrow and approaches a Dirac delta function. 
When this happens, the estimated spectrum converges to the true spectrum. 

In filtering and data adaptive techniques, the spectral smoothing effect 
is not so obvious. It is introduced indirectly when the data function is 
smoothed and processed to obtain estimates of the filter coefficients or 
autoregressive coefficients. Data adaptive spectral estimators such as 
the maximum entropy estimator are superior to the B lackman-Tukey method in 
their ability to minimize the effects of spectral smoothing and to give 
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better spectral resolution. This improved resolution is due to the fact 
that the adaptive methods utilize all of the data while the B lackman-Tukey 
method is restricted to about 5 to 10 percent of the total lag values. 

The reason for limiting the useful range of lag values in the Blackman- 
Tukey method is to minimize the effects of statistical variability in the 
esti mate. 

The stati sti cal variabi I itv of the estimate is caused by the finite 
observation time for the data function. The time autocorrelation function 
definition for power spectra is valid only if the process is both station- 
ary and ergodic. The definition of the time autocorrelation function 
depends on an infinite time. If the data function is observed for a 
finite time, the time autocorrelation function is only an esti mate of the 
"true" function. This introduces statistical variability into the esti- 
mated autocorrelation function that directly carries over into the esti- 
mate of the power spectrum. The effects of statistical variability can 
be reduced only by making many observations of the process to obtain- and 
"average" autocorrelation function. A large number of truncated estimates 
of the autocorrelation function will sufficiently reduce the effects of 
statistical variability but will have no effect on the spectral smoothing 
problem. Spectral smoothing can only be reduced by increasing the" length 
of time the samples are observed. 

If an observed process is not stationary, most spectral theory and 
commonly used spectral estimates will not apply. The power spectrum of 
the process will be a function of time and can only be obtained from the 
ensemble definition of the autocorrelation function. If the time variation 
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is slow enough so that the spectrum is essentially constant for the time 
interval of observation, a "quasi-stationary" estimate can be obtained. 

In this instance, the deleterious effects of a short observation time 
must be traded against those of a time varying spectrum. The display of 
time varying spectra can be accomplished by a power spectral density pro- 
file plot (Ctnes and Enochson, 1972, p. 415). 

The multitude of effects involved in the preprocessing of the data 
function, the sampling process, and the intrinsic spectral estimation 
errors must be collectively considered in the process of method selection. 

D. M ethod Selection 

Method selection should begin with a detailed study of the physical 
process that generates the data function. The purpose of this initial 
study is to discover and incorporate as many constraints as possible into 
the process model. The constraints will help to improve the accuracy 
and/or efficiency of the esttmator and often mean the difference between 
success and failure. After the constraints are determined, the next step 
is to see how the various possible estimators compare in efficiency, 
accuracy, bias, and other appropriate characteristics. For these com- 
parisons, the user must define a criterion of goodness and determine what 
estimator is "best" for his application. This detenni nation will often 
involve a detailed mathematical analysis of the estimator coupled with 
several tests using hypothetical data samples from a "known" process. 
Finally, once an estimator is chosen, the spectra produced by this esti- 
mator must be interpreted as to its accuracy and ability to minimize such 
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effects as aliasing, leakage, and spectral smoothing. 

It is very difficult to develop a step-by-step procedure for the 
spectral analysis of data. Each problem presents its own unique set of 
boundary conditions and each experimenter must choose a criterion of 
goodness that is suitable for his purpose. Spectral analysis "by the 
numbers" is not possible under these circumstances. Some general guide- 
lines are possible if the reader is willing to accept loosely defined 
boundary conditions and criteria of goodness. These guidelines will now 
be presented along with an examination of the more fundamental processes 
involved in spectral estimation. These fundamental characteristics are 
discussed in Section E of this chapter. 

A flowchart of some commonly used spectral analysis schemes is 
shown in Figure 9D-I. The top flowchart shows the possible theoretical 
analysis methods for the theoretical data function, s(t). There are 
essentially three ways of obtaining a spectral function. First, for a 
periodic function, the amplitude spectrum is determined by the complex 
amplitude coefficients of the Fourier series. The spectrum is bandlimited 
and discrete for all real data functions. The amplitude spectrum can be 
used to reconstruct the original data function. Second, the spectrum of 
a single time pulse can be specified by taking the Fourier transform. The 
resulting amplitude spectrum is both continuous and infinite. As before, 
the time pulse can be reconstructed by taking the inverse Fourier trans- 
form. Third, for random processes, the power spectrum is defined as the 
Fourier transform of the autocorrelation function of the process. For 
many theoretical data functions that represent sample functions of a real 
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process, the amplitude spectral function is not defined hecause the Fourier 
transform of the data function does not exist. For these functions, the 
original time function cannot be reconstructed from either the autocorre- 
lation function or the power spectrum. The autocorrelation function def- 
inition for the power spectrum is widely accepted as the most useful for 
random processes and can also be applied, if one accepts the use of the 
Dirac delta function, to periodic functions and single time pulses. 

The bottom flowchart in Figure 9D-I shows several commonly used 
spectral estimators and their most important processing steps. The 
B lackman-Tukey method, the data adaptive techniques, and the autoregres- 
sive spectral estimator all use the autocorrelation function definition 
for the power spectrum. The recursive digital filter methods defines the 
power spectrum by computing the power output of the filter. The direct 
Fourier transform method defines the power spectrum by computing the 
absolute magnitude squared of the complex Fourier coefficients obtained 
by taking the digital Fourier transform of the empirical data function. 

A typical empirical data function is made up of 2N +1 samples of the 
theoretical data function. Since this time series is limited both in the 
total number of samples and the length of observation time, it can be 
exactly represented by a Fourier series with 2N + 1 complex amplitude co- 
efficients (double sided spectrum). The coefficients can be used to de- 
termine both the amplitude spectrum and the power spectrum. The accuracy 
of this estimate will depend on the nature of the "true" data function. 

Likewise, this same data function can be represented by an autore- 
gressive series of any order from order one to order 2N + 1. The power 
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spectrum can be estimated by using the autoregressive spectral estimator 
or the autoregressive series can be used to generate the discrete auto- 
covariance function and then the discrete Fourier transform used. This 
scheme can be applied to any truncated and discrete data function. The 
accuracy of the spectral estimate will again depend upon the nature of 
the "true" data function. 

In still another way, this empirical data function can be used to 
determine the estimated autocovariance function for use in the Blackman- 
Tukey method of spectral estimation. The accuracy of this method will 
depend upon the amount of data available for obtaining the autocovariance 
estimate and the window function selected for smoothing. As the amount 
of data becomes large and the observation time increases, the Blackman- 
Tukey estimate will converge to the "true" autocorrelation function def- 
inition of the power spectrum. This property of the B lackman-Tukey method 
makes it the "best" estimator in terms of adherence to the autocorrelation 
definition of power spectra. 

It may be difficult for the reader to accept the fact that these 
methods may all be considered "exact" for the purposes of representing 
the empirical data function. In this sense, any of the methods could be 
used to defi ne the spectrum of an empirical data function. The only way 
to validate a particular method as being the "best" estimator is to apply 
some a priori knowledge about the process that generated the data. In 
other words, if absolutely nothing is known about the process that gen- 
erated the empirical data function none of the spectral esti mators can 
be selected as a "best" esti mate. 
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The a priori knowledge needed to select a best estimator does not 
have to be very detailed or specific. Usually, a very general assumption 
can be used that will help. As an example, we consider the assumptions 
needed to justify the more common spectral estimators. To use the auto- 
regressive spectral estimator it is necessary only to assume that the 
data function can be represented by an autoregressive series because the 
process is autoregressive. This assumption is surprisingly good for many 
practical data functions. The digital Fourier series representation is 
exact if the data function is assumed to be periodic and if the period of 
observation is exactly equal to one Fourier period. The maximum entropy 
estimator is valid if it is assumed that the process is autoregressive 
and that the resulting spectrum has maximum spectral entropy. An addi- 
tional boundary condition on this estimator is that the autocorrelation 
function and power spectral density function be exact reciprocal trans- 
forms (no window function). 

The B lackman-Tukey estimator differs from most of the other estimators 
in that it converges to the "true" spectrum for random processes as in- 
creasing amounts of data are taken. This circumstance is brought about 
because of the choice of the definition for the apparent autocovariance 
function. The assumptions required for validating this method are: 

1. The Fourier transform of the modified apparent autocovariance 
function is a respectable estimate of the smoothed valued of the 
"true" spectrum. 

2. The statistical variability of the estimate is small enough. to 
be acceptable. 
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In theory, the B lackman-Tukey estimator is capable of giving the best 
spectral estimate when the data function is well-sampled. 

When nothing is known about the process that generated the data 
function, it is safest to select an estimator that requires the weakest 
assumptions about the process. Recent work by researchers in geophysics, 
astronomy, and other scientific areas that require the tools offered by 
digital spectral analysis has shown that the maximum entropy spectral 
analysis method offers higher spectral resolution and improved estimation 
accuracy over previous methods. If large amounts of data are available, 
the B lackman-Tukey method can also be successfully used. 

Figure 9D-2 is a data processing flow diagram that illustrates how 
data might be processed to obtain a spectral estimate. The diagram is 
based on the consideration of several characteristics of the spectral 
analysis problem. These are: 

1. The process model. 

2. Discrete or continuous output. 

3. Amount of available data. 

4. Sampling rate* 

5. Desired accuracy of estimate. 

"Real data" represents the actual signal output from the process. If it 
is continuous, an analog spectrum analyzer can give the desired spectrum. 

If the output is analog but has a finite duration, the data record will 
usually be sampled and preprocessed. The proper experimental design should 
be observed so that the finite record is sampled above the Nyquist rate 
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and pref ilterirfg should be used. If the data output are already in the 
form of a time series, the experimenter cannot exercise control over the 
sampling process. If the data are undersampled, the spectrum of the time 
series will be aliased and spectral accuracy may be seriously degraded. 

If the data is periodic or "almost" periodic, it can be processed 
using the amplitude spectral estimator. A knowledge of the period of the 
function allows the use of some strong model constraints that can improve 
the statistical variability of the estimate by reducing random effects 
due to sampling or noise. If the data is not periodic or if the noise 
level is so high as to make the data function appear to be random, the 
data must be treated accordingly. 

Figure 9D-3 shows one possible scheme for selecting a spectral esti- 
mator for the empirical data function obtained from a random process. 

The decision as to what kind of estimator to use essentially depends upon 
how much is known about the process model and how much data is available 
for processing. A selection of possible estimators is shown. If a large 
amount of data is available for analysis and the process model is unknown, 
the B lackman-Tukey method can be used to give the most accurate estimate. 
The computational efficiency will be considerably reduced from the direct 
FFT method but this sacrifice may be desirable if the experimenter needs 
the improved estimation accuracy. 

If large amounts of data are available, there is a scheme that appeals 
to the author that is not shown in the diagram. The scheme essentially 
would be used when high accuracy estimates are needed. The B lackman-Tukey 
estimator would be used to make iterative calculations of the power spec- 
trum as the amount of data is increased for each estimate. The power 




Figure 9D-3. Data Processing Flow Diagram for Illustrating a 
Possible Scheme of Spectral Analysis for a 
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spectrum would be seen to converge and the rate of convergence and the 
variance of the final estimate would give a measure of the statistical 
variability of the estimate. This variability could be traded off against 
the resolution of the selected spectral window to give a user defined 
"best" estimate. The scheme can be described as the use of smaller sub- 
sets of data to obtain an estimate of the variance of the spectral esti- 
mate that used all available data. The author has not seen this scheme 
developed in the literature and has not had time to develop it in this 
research. Further study could be done to develop criteria for evaluating 
the limits on statistical variance and spectral resolution. 

The ultimate decision about the use of a particular spectral estima- 
tion scheme must be made by the user after serious consideration of the 
need for the estimate and the scientific consequences or conclusions to be 
made from the estimate. The occasional or casual user of spectral analysis 
and spectral estimation methods does not usually have the time to devote 
to a detailed analysis of possible estimation schemes. As a consequence, 
the casual user should be warned against placing great scientific impor- 
tance on the results of the estimate. He should use wel I- developed esti- 
mators as discussed in this chapter or as given in Blackman and Tukey 
(1958) or Otnes and Enochson (1972). Any presentation of an estimated 
spectrum and scientific conclusions based on that estimate should be well- 
supported by a discussion of the details of the estimation scheme used. It 
is especially important to list all assumptions necessary to make the esti- 
mator val id. 
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It would appear to this author that maximum entropy spectral analysis 
will become an increasingly important technique in future years. Its a- 
daptibility to the data and its improved spectral resolution over other 
methods should establish it as the method most recommended to the casual 
user. It is especially useful when the amount of experimental data is 
limited. At the present time, the practical implementation of this tech- 
nique is not well-discussed in the literature. The reader is referred to 
Chapter XV and Appendix III for a further discussion. 

The experienced or serious user of spectral analysis theory and theory 
of spectral estimators must devote a considerable amount of time to the 
study of estimation theory and the criteria of goodness associated with the 
selection of a "best" method. He must especially be familiar with the 
strengths and weaknesses of each method and the attendant assumptions. In 
most cases, more than one spectral estimator should be used to analyze the 
empirical data function of a real process. The use of more than one esti- 
mator can help to reduce the effect of intrinsic estimation errors and 
provide more confidence in the estimate. 

A typical scheme for analyzing the spectrum of an unknown process is 
shown in Figure 9D-4. This scheme employs more than one kind of estimator 
and illustrates what the author considers a reasonable approach to a de- 
tailed spectral analysis study. A theoretical data function is prefiltered, 
sampled, and preprocessed to obtain an empirical data function. This em- 
pirical data function is used to do a spectral study of the process. Since 
the properties of the process are unknown, a process model cannot be de- 
veloped and estimation techniques that do not depend heavily on a particu- 
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lar model should be used. First, the spectrum is estimated by the direct 
FFT method. This estimator is very computationally efficient and can 
provide a good first idea of the spectral shape. If the data is sufficient 
to warrant its use, the B lackman-Tukey estimator should be tried as it will 
give an estimate that converges (In theory, the direct method cannot). If 
the spectrum contains spectral peaks that may not be resolved, the spectral 
resolution can be improved by using the maximum entropy method. The spec- 
trum can be computed by either the FFT or by using the autoregressive 
spectral estimator. To reduce computation time, the autoregressive spec- 
tral estimator need only be evaluated at frequencies near the spectral 
peaks. The results of these estimations should be compared and additional 
processing applied as needed. 

If the data is very limited, it appears that the use of the maximum 
entropy spectral estimator is the best choice for obtaining maximum spec- 
tral resolution. A summary of the fundamental properties of spectral 
estimators will help to illustrate their applicability and help the user 
to make a wise selection. 

E. Fundamental Characteristics of Spectral Estimators 

Spectral estimators are usually developed by starting with the various 
theoretical definitions for a spectral function and applying suitable es- 
timations and numerical approximations. A spectral estimator is usually 
designed to minimize the intrinsic estimation errors caused by finite data 
sampling. The various methods appear to differ because they represent 
different viewpoints concerning the "best" way to represent empirical data. 
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These different viewpoints usually are the result of selecting different 
criteria of goodness and starting with different a priori assumptions. By 
carefully analyzing the commonly used spectral estimators, one can reach 
the conclusion that they do not represent entirely different approaches to 
the estimation problem. Because they are all derived from the same theo- 
retical basis, the various estimators share many common characteristics. 

When developing a suitable spectral estimator, there are essentially 
three commonly held viewpoints concerning the best way to treat the problem 
of finite data. These viewpoints have been developed by the author from 
reading the literature in this area and they may not represent the break- 
down that another author might choose. This author believes that this 
separation adapts itself nicely to the discussion of method selection. 

The three methods are: 

1. "Lag windowing" of the estimated autocorrelation function. 

2. Smoothing and extrapolation of the autocorrelation function. 

3. Recursive digital filtering. 

The use of a lag window function to give a smoothed estimate of the power 
spectral density function represents the approach taken by Blackman and 
Tukey (1958). The use of smoothing and extrapolation to obtain an estimate 
of the autocorrelation function is the method chosen by those who prefer 
data adaptive methods which do not have fixed window functions. Data a- 
daptive methods such as maximum entropy and maximum likelihood (Lacoss, 

1971) smooth and extrapolate using their respective criteria. The trun- 
cated autocorrelation function can also be smoothed and extrapolated using 
purely mathematical means such as model fitting with some exact function. 
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This can be done using the straightforward techniques common to numerical 
analysis (Chapter III) or by using more exotic orthonormal expansions such 
as angular prolate spheroidal wave functions (Veltman et al., 1972). Re- 
cursive digital filtering gives an estimate of the frequency spectnom by 
computing the average power over the bandwidth of the filter. The filter- 
ing method adds the flexibility of choosing the bandwidth of the estimate 
(within certain limits) but still suffers from problems such as leakage 
and computational efficiency. Recursive filtering has the tutorial ad- 
vantage of being able to represent a wide variety of discrete spectral 
estimators including the important maximum entropy estimator. 

The use of a lag window and the autocorrelation function definition 
of a power spectrum is treated in great detail in Blackman and Tukey (1958). 
A brief summary of this method showing how the use of a window function 
reduces the effects of statistical variability and produces a smoothed 
estimate of the spectrum will now be presented. In discussing the Black- 
man-Tukey method we will use the autocovariance function instead of the 
autocorrelation function. 

An unbiased estimate for the autocovariance function (Appendix I, 

Part I ) is defined by 

+<v M>/2 

C°(t) = ^ " -j - I J~x( t - T/2)x(t+T/2)dt (9E.I) 

-(V M)/2 

where x(t) is a sample record of the process that has been observed for a 
time C q (t) is called the apparent autocovariance function because it 
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is obtained from only one sample realization of the process. For each 
observed record, x(t), there will be a corresponding function, CO(T). 
Because C (t) is an unbiased estimate, its expectation value for lag times 
between ±T^ will be equal to the "true" autocovariance function, C(T). If 
several sample records are available, the corresponding apparent autoco- 
variance functions can be averaged to give a better estimate. Lag times 
near zero will show the least statistical variation because the most data 
is used for the estimate. As the lag value approaches ±T^ the amount of 
data used for the estimate becomes very small and the statistical variance 
becomes very large. 

To avoid the use of lag values with large variances and to reduce 
problems associated with the truncation of the autocovariance function, 
Blackman and Tukey chose to multiply the apparent autocovariance function 
by a lag window function, D(T). The resulting function is called the mod- 
ified apparent autocovariance function: 

<?(t) = D(t)C o (t) (9E.2) 

Although it is purely arbitrary, the lag window function is usually chosen 
so that it will give the most weighting to lag values near zero and 
smoothly reduce the weighting factor to zero at lag values approaching 5 
to 10 percent of the total record. 

For this discussion, <?(t) will be called the estimated autocovariance 
function because it is this function that is used to obtain the estimate 
of the power spectrum. To examine the accuracy of this estimate, we first 
take the expectation value of <$(t) as given by: 
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e{<*(t)} =D(T)E(C 0 (T)} = D(T)C(T) M<T n (9E.3) 

Since, by definition, the expectation of C (T) is equal to the "true" 
autocovariance for lag values between +T the expectation of £(T) is 
equal to the "true" autocovariance multiplied by the chosen lag window 
function. 

The estimated power spectral density function is obtained by taking 
the Fourier transform of the expectation value of t(T). The result is: 

^1e{($(t)}} = E{^{<*(t)}} = E £#(f ) } (9E.4) 

F^(f ) is the Fourier transform of one particular estimate of the autoco- 
variance and E{^(f) } is the expectation value of all possible realizations. 
The Fourier transform of the expectation value of <*(t) is equal to the 
Fourier transform of the product D(T)C(T) as given in Equation 9E.3. The 
transform of the product is the convolution of the transforms of D(T) and 
C(T). The final result is that the expected value for all possible real- 
izations of the estimated spectrum is equal to the convolution of the 
"true" power spectrum with the spectral window function Q(f) 

A +C ° 

E{P(f)} = J Q(f - X)P(\)dX (9E.S) 

_cc 

where Q(f) is the Fourier transform of the lag window function (Appendix I, 
Part I). The best we can do with a finite data function is to give a 
smoothed estimate of the "true" power spectrum. Increasing the number of 
data records will help to reduce the problem of statistical variance but 
only an increase in observation time can help to improve the spectral 
resol uti on. 
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Under certain circumstances, it is theoretically possible to solve 
the convolution integral (deconvolution) to give the "true" power spectrum, 
P(f), when the spectral window function, Q(f), is known. For most prac- 
tical analyses this is not done because, with only a few sample records, 
the error due to statistical variance is great enough to more than offset 
any improvement derived from deconvolution. 

A simple example will be given to illustrate the smoothing and leakage 
problems associated with the use of a lag window. For simplicity, we 
choose a periodic sample function, x(t), made up of three sinewave com- 
ponents. Many, sample observations of x(t) are taken and the corresponding 
apparent autocovariance functions computed. These values are then averaged 
to produce a close estimate of the "true" autocovariance function for lag 
values between ±T |^|* This averaging reduces the statistical variance for 
the apparent autocovariance function. The "true" autocovariance is not 
known for lag values greater than TN because the sample function was not 
observed for time intervals greater than this. 

The average (or expected) value for the time autocovariance function 
is shown in Figure 9E-I. This truncated function is multiplied by a 
rectangular window function to produce the estimated autocovariance func- 
tion, C(t). The width of the window function must be chosen narrow enough 
to minimize the effect of statistical variation in the observed sample 
functions. Blackman and Tukey recommend that the width of the lag window 
be no more than 5 to 10 percent of the total lag record. In our example, 
the width is much wider because the statistical variance has been reduced 
by averaging multiple observations. For this example and others to follow, 
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continuous functions will be illustrated. Usually the actual data 
processing will be done digitally but the continuous functions more clearly 
illustrate the results without the additional complication of sampling. 

The spectral functions associated with the B lackman-Tukey method and 
this example are shown in Figure 9E-2. The effects of windowing and 
truncation can be illustrated by comparing the "true" spectrum, p(f), with 
the estimated spectrum, ( f ) . The estimated power spectrum has been ob- 
tained by convolving the spectral window function, Q(f), with the "true" 
spectrum, P(f). The spectral window function is a sine function obtained 
by taking the Fourier transform of the rectangular lag window function. 

The estimated spectrum is the superposition of many sine functions. Each 
line spectrum in P(f) will generate a sine function of corresponding 
amplitude in the estimated spectrum, l^(f). These sine functions are shown 
separately in the last plot of Figure 9E-2. Spectral smoothing has caused 
the two closely spaced components at f| and to be smeared together. The 
smaller spectral component at f ^ has been obscured by this loss of spectral 
resolution and most likely would go undetected. Also, the spectral com- 
ponent at f^ has been smeared out and leakage has caused considerable error 
in the estimated amplitude of the lower frequency components. Not shown 
is a dc component in the spectrum caused by the net average value in the 
estimated autocovariance. 

The window function has also introduced some negative power spectra. 
"Negative power" is not possible physically and its presence in this esti- 
mation routine is one of the major complaints of the users of this method. 
The generation of negative components in the power spectrum can be reduced 
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Figure 9E-1. The Estimated Autocorrelation Function Obtained from a 
Rectangular Lag Window 



Figure 9E-2. Example Spectral Functions that Illustrate the 
B lackman-Tukey Method 
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by the proper selecti.on of a lag window function. The rectangular window 
function is seldom used because it causes such a severe leakage problem. 

It does, however, have the best spectral resolution. Other window func- 
tions such as the Hanning window or Hamming window (Otnes, 1972, p. 258) 
have much smaller leakage with only a small sacrifice in spectral resolu- 
tion. 

In many ways, the B lackman-Tukey method of spectral analysis can be 
viewed as a "fix-up" approach for handling a "difficult" apparent autoco- 
variance function. They use essentially a three step approach to obtain a 
spectral estimator. First, they define one estimate of the time autocor- 
relation function that they call the apparent autocovariance function, 

C o (t). While this estimator has desirable theoretical properties, it also 
has serious practical shortcomings. Second, the practical shortcomings of 
this estimator "are minimized by" using a lag window to obtain a second es- 
timator for the "true" autocovariance function. This second estimator, 
which Blackman and Tukey call the modified apparent autocovariance function 
and the author calls the estimated autocovariance, is the function that 
must represent the "true" autocovariance function for spectral estimation 
purposes •. T hi rd, the Fourier transform, of the estimated autocovariance 
function, is used to obtain the estimated spectrum. This three step 

approach has good" and bad aspects that will now be summarized. 

For theoretical purposes, the B lackman-Tukey definition of the apparent 
autocovariance function has many desirable characteristics needed for a 
good estimator. It converges to the "true" function as the record length 
becomes infinite and hence is unbiased. Also, its expectation value is 
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equal to the "true" funct.ion for lag values between ±T^ • 

For practical purposes, these desirable characteristics are often 
overshadowed by the problem of statistical variation. This can introduce 
a "de" component not present in the "true" function. It can also make the 
estimate very poor for large values of lag. This happens because the es- 
timator (Equation 9E.I) uses very little data for lag values approaching 

±Tn* 

A lag window function can be used to reduce the effects of statistical 
variance by removing the larger lag values. This effectively reduced the 
statistical variance in the estimated spectrum but it also introduces 
spectral smoothing effects. The selection of a suitable lag window be- 
comes a tradeoff choice between spectral resolution and statistical vari- 
ation. A lag window removes valuable data that, although it may have a 
large variance, could still be used to give some improvement in the spec- 
tral estimate. 

A technique that avoids the problem of choosing a lag window function 
and that attempts to use all of the observed data has a lot of practical 
appeal. Such techniques come under the broad category of data adaptive 
techniques and apply methods of smoothing and extrapolation to obtain an 
"acceptable" estimate for the autocovariance function. Conceptually, 
smoothing and extrapolation are more difficult to justify than the use of 
a lag window. A criterion of goodness must be defined and its effect on 
the accuracy of the spectral estimate must be determined. It is entirely 
possible that an acceptable rational for snoothing or extrapolation may 
lead to a very unacceptable spectral estimate. 
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Generally speaking, two different viewpoints have been adopted con- 
cerning the "best" way to achieve smoothing and extrapolation. The first 
adopts a purely mathematical viewpoint and proceeds on the basis of well 
established numerical techniques for fitting numerical data with exact 
functions. This approach is often called a parameter free procedure (Velt- 
man et al., 1972) because it does not depend upon any a priori assumptions 
about the observed process or the resulting data. The resulting spectral 
estimate is independent of any model for the process. The second viewpoint 
insists that an estimate of the "true" autocovariance function should be 
developed by assuming a reasonable and acceptable model for the process 
or the data. 

The parameter free procedure for smoothing and extrapolation is not 
well-suited for spectral analysis applications because it does not allow 
for the input of important information that may be known about the process. 
Also, numerical analysis techniques employ criteria of goodness that can- 
not be easily related to the determination of the accuracy of the spectrum. 
In practical applications, the addition of known information about the 
process can be used as a powerful tool to improve the spectral estimate. 

For these reasons, it is usually better to choose a method based on some 
model. 

Estimation procedures that employ some type of process model do not 
have to be as restrictive as one might imagine. It usually turns out that, 
for purposes of general spectral analysis applications, the process model 
is designed to have wide applicability and only very modest restrictions 
on the data function are- needed. If a considerable amount of model infor- 
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mation is known about the process, the user is well-advised to design a 
spectral estimator that incorporates all of this information. This un- 
usual situation precludes the use of a "standard" procedure. 

Process models that have been very successful for obtaining smoothed 
and extrapolated autocovariance functions are those used in autoregressive 
spectral estimation and maximum entropy spectral estimation. These tech- 
\ niques operate directly on the observed data function to produce both the 
estimated autoregressive coefficients and the estimated autocovariance 
function. The power spectrum can then be estimated by using either the 
autoregressive spectral estimator (Chapter XIV) or by taking the Fourier 
transform of the estimated autocovariance function. 

For the autoregressive estimator, it is assumed that the observed 
data can be modeled by an nth order autoregressive series. Once the 
series coefficients have been determined by some appropriate smoothing 
and fitting technique, they are used in the spectral estimator. In the 
maximum entropy method, it is assumed that the process produces a spec- 
trum that has max im um spectral entropy subject to the constraints that 
the process can be modeled by an autoregressive series and that the esti--. 
mated. autocovariance and the estimated power spectral density form an 
exact transform pair (no window function). 

The estimated autocorrelation function produced by these methods 
satisfies all of the theoretical requirements set forth in the discussion 
on frequency domain analysis in Chapter IV. It is discrete but it obeys 
a well-defined mathematical equation and is defined for all lag values. 

This estimate can be considerably different from the one obtained using 
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the B lackman-Tukey method. It is now generally accepted that the maxi mum 
entropy method of spectral analysis has a spectral resolution superior to 
that of the B lackman-Tukey method. 

It is difficult to see just how these techniques develop a smoothed 
and extrapolated estimate unless the autoregressive series approach to the 
characterization of a random time series is studied in considerable detail 
This is not necessary for our purposes and the interested reader is re- 
ferred to a book by Koopmans (1974). It is important for the reader to 
realize that these techniques can be described by the mathematics associ- 
ated with the recursive digital filter. For example, the maximum entropy 
algorithm can be described as a method for the design of a filter that 
prewhitens the input spectrum. The filter coefficients can then be used 
to obtain a spectral estimate (ulrych, 1972b). This is also commonly re- 
ferred to" as prediction-error filtering (Peacock and Treitel, 1969). The 
Burg algorithm for determining the maximum entropy spectrum is a recursive 
procedure for calculating the prediction-error filter and the error power 
of the prediction (Smylie et al., 1973). 

The maxi mum entropy method, in particular, assumes that the process 
can be accurately described by an all-pole filter model (a "purely recur- 
sive" digital filter) and that the error power is minimized by a least- 
square-error criterion. These assumptions also apply to the smoothed and 
extrapolated estimate of the autocovariance function. 

An excellent discussion of smoothing and extrapolation techniques for 
estimating autocorrelation functions with several examples is given in a 
paper by Veltman et al. (1972). 
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A third method of developing a spectral estimator involves the use 
of recursive digital filtering. 1 1 was stated in the previous discussion 
that a recursive digital filter could be used to obtain a maximum entropy 
spectral estimate by using the concept of prewhitening. It can also be 
used in the more classical sense as a digital filter operating directly on 
the observed data. 

An analog spectrum analyzer sweeps a narrowband filter through the 
spectrum of the incoming signal and the detected power output of the filter 
is displayed as a function of frequency. For digital spectral analysis, 
the digital filter must be "stepped" through the desired frequency range 
and the "power output" determined at each step. At each new frequency step, 
the filter coefficients must be changed and the data processed through the 
filter. The method operates as though the actual time function were passed 
through an electrical filter and the resulting output detected to determine 
the average spectral power over the passband of the filter. Adopting, this 
viewpoint wi II help the user to understand that the digital filter has all 
of the "real" problems associated with the analog filter. 

There are several problems associated with the use of digital filter- 
ing for spectral analysis. The transient response or "ringing", of the 
filter and its finite bandwidth reduce the accuracy of the spectral esti- 
mate. The transient response is especially troublesome if the data record 
is so short that the filter never reaches "steady state". The required 
modification of the filter coefficients for each frequency step and the 
repeated running of the data through the stepped filter may require large 
amounts of computation time. Finally, the user must develop a filter 
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and a criterion of goodness that is suitable for his purpose and that pro- 
duces an acceptable estimator. 

This last task is probably the most difficult one facing the potential 
user. If he is forced to use one of several "canned" procedures, he loses 
the ability to incorporate known information about the process. This is 
a serious disadvantage if model information is reasonably accurate. Unless 
a digital filtering technique has obvious advantages for a particular 
process model, it is usually advisable for the user to select another 
method of analysis. A discussion of the application of digital filtering 
to the estimation of power spectra is given in Otnes and Enochson (1972). 
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X. HILBERT SPACE VECTOR FORM IJLATION OF 
DIGITAL FOURIER TRANSFORM ANALYSIS 


When this research was started and the mathematical forms for Fourier 
series and Fourier analysis were being investigated, the author noted the 
mathematical similarity between the Fourier analysis equations and those of 
Hilbert space vector analysis. Flaving just completed a quantum mechanics 
course, it seemed worthwhile to use the now familiar Hilbert space vector 
analysis to investigate the properties of digital Fourier analysis. The 
vector formulation that will follow was worked out in detail by the author 
and proved to be extremely useful in some of the subsequent mathematical 
derivations. Since that time, the author has discovered other references 
to this Hilbert space - Fourier analysis analogy but has not seen the 
mathematical details presented. The material that will now be presented 
was subsequently used to derive the spectral mixing formula and to describe 
many of the properties of digital Fourier analysis. 

The complex representation of a Fourier series is written as 
+® +j 2TT nx 

f(x) = E C e T (10.1) 

n=-® n 


which has complex amplitude? defined by: 


C 

n 


2tt 1 (»+T!2 -J T 
= C|-n) = j J-|- j 2 e *(x)dx 


( 10 . 2 ) 


The function f(x) is periodic with a Fourier period of T. The independent 
variable, x, ranges over +■» but the pri nci pal range of x is ±T/2. Since 
f(x) is periodic it is completely specified by its values over the principal 
range of x. Since f(x) is not bandlimited, it may theoretically take an 
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infinite number of complex amplitudes to describe it. 

In Hilbert space vector notation, the complex representation of a 
Fourier series is written as: 

f(x,T) = ? a a (x,T) = (<* n ,f)ar n (xJ) (10.3) 

n=-® n n n=-® 1 

The orthonormal basis set of eigenvectors is given by 

+j 2TT nx 

Q^(xJ) = e T (10.4) 

where the vector "direction" is specif i .ed by the parameter nIT. The 
Hilbert space vector f(x,T) is an infinite linear combination of eigen- 
vectors a n (x,T). The complex expansi.on coefficients which are the eigen- 
values of f(x,T) are defined by the vector inner product as: 

1 i+T!2 * 

a = O ,f) =1 I a (x)f(x)dx (10.5) 

n n - 772 n 

The "direction" of the Hilbert space vector is analogous to the Fourier 
period and the complex expansion coefficients are analogous to the Fourier 
complex amplitudes. 

The vector norm for Hilbert space vectors is given by: 

1 i-HT/2 * « 2 

(f,f) =t ) f (x)f(x)dx = £• lanf (10.6) 

T -772 n=-® 

The derivation of the complex coefficients from the definition of the 
vector inner product will now be done because it is important for later 
work. The vector .inner product given in Equation 10.5 is written with 
Equation 10.3 substituted for f(x,T) as 

l +T/2 * +® 

'n *' T 


(10.7) 
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where the dummy index k has replaced n. Interchanging summation and inte- 
gration and grouping the variables in x, we obtain the expression: 

+- , +T/2 

I = S (a ,f) - f or (x)ql (x)dx (10.8) 

n k=-® * T -T/2 n k 

The integral in (10.8) represents the vector inner product of the eigen- 
vectors and but since these form an orthonormal basis set, their 
inner product is the Kronecker delta function. These steps are summarized 
as: 

'n = ^'V f)( V\) * „ \ < “k’ f)6 nk < l0 - 9 > 

k=-<® k=-® 

The Kronecker delta operating on the inner product (c^jf) contracts to 
give the inner product (o? n ,f ) and this was the original definition of the 
complex coefficients: 


'n -( V f) “ a n 


( 10 . 10 ) 


The derivation of the vector norm is shown in the same way by starting 
with the definition given in (10.6). Equation 10.3 is substituted for 
f(x,T) to give: 

1 ,+1/2 +*> * * +” 

(f,f) =-T L,_ E (or ,f) <* (x) S (a ,f)a (x)dx (10.11) 

-T/2 n s *® n n * < 

Grouping variables in x and interchanging summation and integration gives: 

+*> * +* 1 (d-T/2 * 

(f,f) = X (o- ,f) 2 (a ,f) - j or (x)a (x)dx ( 10 . 12 ) 

n=-« n k=-« K 1 -T/2 n K 


As before, the integral defines the vector inner product which 

reduces to the Kronecker delta to give: 
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| ^ ^ j Q3 

(f,f) = 2 (or ,f) I (a. ,£> 6. 

n= -® n k=- c ° 


* 

nk = E(a n ,f) (a n ,f) (10.13) 
n=-® 


The vector inner products, (a n ,f)> define the complex amplitude coefficients, 
and the final result becomes: 


-y | rr 

(f,f) = 2 a n a n = 2 |a 

n=-» n=-=> 


n 1 


(10.14) 


These same equations which have been derived in Hilbert space notation 
will also be derived using the notation for Fourier series given in 
Appendix II. The physical and mathematical significance of these various 
quantities as related both to the Fourier series and Hilbert space vectors 
will be presented. 

A periodic function can be represented by a series expansion around 
some vector basis set. The complex representation of a Fourier series is 
one such expansion that can be written in the form: 




+ J 2 - T h 


X 


f(x,T) = 2 (<* ,f)ff (x,T) = I c e T 

' n n ’ _ n 

n=-co a=-*> 


(10.15) 


The orthonormal basis set of eigenvectors for a Fourier series expansion 
is defined by Equation 10.4. The complex expansion coefficients, a n , are 
represented by the Fourier complex amplitudes, c n> and both are defined by 
the vector inner product, (o^f) .. 

The vector inner product can be used to calculate the complex Fourier 
amplitudes and also to simplify mathematical derivations involving Fourier 
series. It is this latter function that the author found very useful. To 
show that the complex amplitudes can be derived from the vector inner 
product of the eigenvector operating on the state function, ve evaluate 
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the following integral: 


+.-U&X 4- +j 2 D k 

=IS"e T [ ste T ]dx 

n T — CO ico K 

(10.16) 

„ . 2tt . 2tt t 


+°° .1 S+TI2 -j ^nx +j ^kx 

= S cl - e e dx 

(10.17) 

k=-«° K 1 -T ! 2 


-t~ 1 ^!2 +AZcT(k-n) 

= Ec, j dx 

k=-« T -77Z 

(10.18) 


The integral in (10.18) is recognized as one form of the Kronecker delta 
function discussed in Appendix II, Equation AZ. 33. Replacing the integral 
with the Kronecker delta function, the contraction of 6^ and gives the 
complex amplitudes: 


+00 


(< V f) - 1 n S ®k 6 kn ■ c n 

n*-oo 


(10.19) 


These complex amplitudes are used to reconstruct the original time function, 
compute the average powet, and determine the discrete amplitude spectrum. 

The vector norm is used to determine the mean-square voltage or 
"average" power in the periodic time function. The vector norm is also an 
alternate representation of Parsaval's theorem. The derivation of the 


vector norm for a Fourier series is summarized below: 

1 +T/2 +» * -j ZrTnx +j ^jr kx 

(f»f) =T S Sene S c,e dx 

- -T!2 n=-® k=_CD K 


. 2tt 


. 2tt. 


= E c 


n=-co k=- oo 


! «/2 -Jfnxtjfloc 
T J, 


dx 


( 10 . 20 ) 


( 10 . 21 ) 


-T/2 


+=> 


+00 


■ HD * 

n . ~ °k ®kn ^ C n C n 

JJ=—CD Q3«CO 


= E c 


( 10 . 22 ) 
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This expression for the vector norm can also be written in the more usual 
form: 

<f,f) = £ |cj 2 (10.23) 

n=-® 

Parseval's Theorem for Fourier transformable functions is given by 
(Cooper, 1971, p. 133): 

S x 2 (t)dt =jz ^IX(w) I dw (10.24) 

For a Fourier series it is (Thomas, 1969, p. 600): 

12 “22 1 /tV772_ 2 

-2 a + £(a+b)=-TO r(t)dt=(f, f) (10.25) 

o n=l n n -T/2 

In Equation 10.25 the sin-cos expansion coefficients are used. If these 
are replaced by the complex coefficients, this expression is equivalent 
to the vector norm equation. To show this we expand the vector norm 
equation using (10.6) to give 

i sr-Voo* = tic i 2 

T -T/2 n=»® 0 

where it is assumed that f (x) =f(x). 


(10.26) 
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XI. SPECTRAL MIXING FORMULA 


A bandlimited time function, x(t), with period Tp can be exactly 
specified for all time by a Fourier series 


•HI +J 2 ¥ nt 
x(t) = £ c e F 

n— M n 


-CO < t < 4-35 


( 11 . 1 ) 


with complex amplitudes: 


c = 


l+T / 2 
1 F x(t)e 


,2TT „ 


dt 


-M £ n 


-T p/2 


( 11 . 2 ) 


The following quantities are defined: 


Tp =The period of x(t) which will be referred to as the 
Fourier period. 

2 TT 

u^, = ^ the Fourier frequency which is also the lowest possible 
frequency component in x(t). 

Co =The average or tide" value of x(t). 

C^ =The ccm^lax spectral amplitude of the nth frequency com- 
ponent corresponding to nu^. 

Mu^ = The highest frequency component in x(t). 

The complex spectral amplitudes are specified by the integral of 
Equation 11.2 but this representation can only be used to develop the 
theory or when x(t) is specified as a mathematical function. For empirical 
data, the integral must be approximated numerically. Suppose we define a 
hypothetical experiment where x(t) is sampled at a rate exactly equal to 
the Nyquist rate and we assume that the total number of samples taken is 
sufficient to completely determine the complex spectral amplitudes, c p * 
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For these conditions, the sampling interval is defined as At =77 7 /(2M + 1) 
and the total number of samples is 2M + 1. This means that the function 
has been sampled for exactly one Fourier period. 

The complex amplitudes will be estimated by using a Riemann sum ap- 
proximation for the integral of Equation 11.2. This approximation can be 
written as follows (Appendix I, Part A): 


2TT 

A i « -v kAt 

I =- At Sx(kAt)e 
n T F k=-M 


(11.3) 


Since the total number of samples taken is 2M + 1, there are a total of 
2M + 1 segments to be summed. If the sampling interval is replaced by, 
bt = TFI(2M +1), the Riemann approximation can be written as: 


m • 2'11 nk 

a 1 -wt I 2TVT+! 

Tn = 2M + 1 Sx(kAt)e J 
k— M 


(11.4) 


Does this Riemann sum approximation for estimating the complex amplitudes 
give accurate estimates for these sampling conditions? This answer can be 
obtained by computing the vector inner product using the estimate. The 
The procedure is the same as that used to derive Equation 10.19. Substi- 
tuting Equation 11.1 for x(kAt) into (11.4) gives: 

• 2rr , „ 2 tt 

A 1 1? 2M + 1 uk tr +j2M±1Lk 

I =2M + 1 2 e ScLe ( n -5) 

n k=-M L=-M 

This expression is modified by interchanging suumation and combining the 
exponents to give: 


4M 


. f i f + yrT< L - n >\ 

n L=?H L2M+1 Ic-M 5 


( 11 . 6 ) 


or by applying trignometric substitutions, the alternate form is: 
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A 

I 

n 


-m 
■■ S c. 
L=-M 


i.feL Cl+2 


M 

I cos 
k=l 


2tt 

2M+ 1 


(L - n)k]} 


(11.7) 


It has been determined that the quantities in the brackets represent the 
K ronecker delta function (Appendix II, Equations A2. 36 and A 1.37). To 
prove this, a computer program was written for the expression in brackets 
and this expression was evaluated for various combinations of the integers 
L and n. The double precision program showed that for integers restricted 
to (L - n) £2M, the representation is exact. This discovery was used to 
conclude that: 


A 

I 


n 


•+M 
£ c. 
L=-M 


L 6 Ln 


c 

n 


( 11 . 8 ) 


In other words, when the time function is sampled in this very special way, 
the Riemann sum estimation for the integral gives the exact values of the 
complex spectral amplitudes. Hence, the digitally computed complex ampli- 
tudes can be used in the series of Equation 11.1 to exactly reproduce the 
original continr.oua time function. 

Since the reproduction is exact for the continuous function, it is 
also exact for the sampled function, x(k£t), If Equation 11.4 is used as 
a defi niti on for the complex amplitudes and Equation 11.1 is written in a 
discrete form, these two equations become a basis for the definition of a 
di qital F ourier transform pair: 


-w 

x(k£t) = See 
n=-M n 


+ J 1 2M + l Uk 


C 

n 



-m 

£ x(k£t)e 

k*-M 


Prr 

3 2M+1 


uk 


(11.9) 


( 11 . 10 ) 
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The digital Fourier series and the digital Fourier transform (Bergland, 

1969) have the same mathematical form. From the data processing viewpoint, 
they are equivalent. 

Discrete Fourier transform (DFT) routines including the fast Fourier 
transform (FFT) are designed to produce M independent frequency components 
and a "dc" term from 2M + 1 samples of a data function. If the ideal 
sampling conditions previously described are achieved, the DFT will give 
exact values for the <^. If these conditions are not achieved, it will give 
only an estimate. When the discrete transform is arbitrarily applied to 
2M+ 1 data values, the resulting discrete ampl i tude spectrum will be such 
that the data values can be exactly reproduced. For this reason, the 
transform pair given above is called an exact transform pai r. In general 
an estimator for the complex amplitudes will not produce a series that will 
give back the exact data values. 

A mathematical examination of the effect of sampling on the estimation 
of a discrete amplitude spectrum has led to the development of the author's 
spectral mixing formula. In this derivation, the vector inner product was 
used to define the complex amplitudes and a Riemann sum was used to approx- 
imate the integral in (11.2). The goal of this analysis was to determine 
the effect of sampling interval and number of samples on the accuracy of 
the estimate. 

Suppose a time function which can be exactly specified by Equations 
11.1 and 11.2 is sampled over an arbitrary observation interval, T^. It is 
assumed that the observer is unaware of the exact value of the Fourier 
period, T ? . It is also assumed that there are 2N+1 samples in the interval 
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and that the arbitrary sampling interval is At. 

First, we. define an estimated amplitude spectral function as the dig- 
ital Fourier transform of the time series {x(kAt) : k=0,±l,±2, •••, +N j: 


£(w) = At T- x(kAt)e 
k=-N 


( 11 . 11 ) 


It should be noted that this estimate produces a continuous function of w 
despite the fact that x(kAt) is discrete. The continuous nature of ft(w) is 
a result of the definition of the estimation procedure. In some cases, 
this continuous estimate can be exploited to produce better results than a 
discrete estimate. 

Second, the complex spectral amplitudes associated with the time series 
above are defined to be: 

W -j^nkAt 

C At S x(kAt)e 'N 

n x n k=-N 


1 ^ 

= 2N + 1 S x(kAt)e 


,• 2iT ,nk 
- J 7N +1 


( 11 . 12 ) 


k=-N 

These complex amplitudes are related to the estimated amplitude spectral 
function for x(kAt) as follows: 


A 

C 


_T N 




T n = ( 2N + 1) At 


(11.13) 


In other words, c is obtained from X(w) by sampling at intervals corres- 
ponding to w n = 2Tm/T {f and dividing by the observation interval. Before 
continuing with the derivation, the following observations about the esti- 
mation process should be made: 
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A ^ S 

Under very special circumstances the c n are exactly equal to the 
"true" complex amplitudes, c , and hence can be used to generate 
the continuous time function x(t) from the samples x(kAt). 

2. For any given data set {x(kAt): k=0,±l,±2, •••, ±N }, the esti- 
mated coefficients c^ specify a Fourier series given by 


x(t) 


4N 
= I 
n=-N 


A 

c e 
n 


, . 2tt 


(11.14) 


which is periodic with period TN and which exactly reproduces the 
sample values x(kAt) . From this we realize that x(t) and x(t) are 
equal at the times t = kAt but, in general, are unequal everywhere 
else in the time domain. For this reason it is concluded that 
there are an infinite number of x(t) functions which will fit a 
discrete data set. Any particular x(t) is completely specified 
by the sampling interval At and the total number of samples, 

2N+1. The desired goal of the estimation scheme is to obtain 
x(t) and its amplitude spectral estimate as close as possible to 
x(t) and ^''{x(t)}. 

3. For 2N + 1 samples of x(t) there are 2N + 1 values of c computed 
in a digital Fourier analysis scheme. In the "true" spectrum 
there are 2M + 1 coefficients. This means that for oversampling 
there may be many more estimated coefficients than "true" co- 
efficients while undersampling produces just the opposite. 


We now proceed to derive the spectral mixing formula. For conven- 
ience, a continuous complex spectral amplitude estimate will be defined as: 
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c(w) — 1 £ x(kAt)e 

2N+1 k=-N 


(11.15) 


where At=T fI /(2N +1). This continuous function gives the discrete coef- 
ficients when sampled at intervals corresponding to nAiu=|? n. 

Tt 

To determine the accuracy of this spectral estimator it is convenient 
to compare the "true" complex amplitudes with the estimate; This may be 
done by evaluating the vector inner product using the estimator. Substi- 
tuting the "true" Fourier series given by (11.1) into Equation 11.15 gives 
the following estimate in terms of the "true" amplitude coefficients: 


K 1 +M +j|£LkAt t 

C(w) = - £ E c_e J r ej^ 

2N + 1 k=-N L=-M l 


(11.16) 


This equation can be manipulated into the form: 




i , * (! * 

£<W) ' , z „ e 


(11.17) 


k»-N 


Using trigonometric substitutions it can also be written'as: 

. , N T 

c(.w> = s,'i i C STi [i+2 


L=-M 


k*l 


m a 

= Z c L 5(Lu^-tti) 

Ir^—M 


(11.18) 


(11.19) 


Equati.on 11.18 is one form of the spectral mixing fonmla. In this 
form, it describes the mixing effect of sampling on the continuous complex 
spectral amplitude estimate. .In another form it will be used to 4escribe 
the spectral mixing effect for the discrete spectral amplitudes, c n . The 
quantity in brackets has a very special significance and will be represented 
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by 6(Lu^-ai). As far as the author is aware, this conc.ept has not been 
previously developed. It will be called the esti mated K ronecker delta 
function because when it is used to describe the discrete case, its func- 
tional behavior is similar to a Kronecker delta function. It could also 
reasonably be called a Dirac delta function except its amplitude is nor- 
malized to unity and it is primarily used in estimating the spectral ampli- 
tudes in the discrete function. 

The estimated spectral function at any particular frequency is com- 
posed of the sum of all the "real" components after they have been weighted 
by the estimated Kronecker delta function. In many ways this weighting 
can be thought of as a convolution of the discrete spectrum with the esti- 
mated delta function. This viewpoint suggests an analogy between this 
■process and the convolution with a window function as e.ncountered in the 
B lackman-Tukey method. The concept of an estimated Kronecker delta func- 
tion can be applied to other spectral estimators. The one derived here is 
valid only for the Riemann sum approximation. 

The "filtering" effect of the estimated Kronecker delta function is 
shown with the help of Figure 11-1. The delta function becomes "peaked" 
about a frequency corresponding to w=Lu^. This weighting tends to "sift 
out" the discrete amplitude corresponding to the observed frequency, uj, 
and deemphasize the other components, When the amount of observed data 
becomes large, the estimated Kronecker delta function converges to the 
"true" function. When the. hypothetical sampling circumstances described 
earlier are achieved, the estimate becomes exact. 
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The spectral mixing formula for the discrete spectral amplitudes is 

obtained by sampling the continuous complex spectral amplitude estimator 

at frequencies corresponding to nAuj = i? n. The resulting equation is: 

ns 



L=-M 


m , N 

2 C L^2 N + l^ 1 + 2 k !l CQS 


2-nk 


2N + 1 V T . 








11 . 20 ) 


The estimated discrete complex amplitudes are the same as those that would 
be calculated by the discrete Fourier transform. If the observation inter- 
val and the Fourier period are equal, Tp =TN' the estimate is exact. 

The discrete form of the spectral mixing formula relates the "true" 
spectral amplitudes, c , to the estimated amplitudes, c , for any given 
set of sampling conditions. The parameters of the sampler are the total 
number of samples, 2N+1, the observation interval, T^, and the sampling 


interval. At 


_Jn 

2N + 


1. The parameters of the input signal are the Fourier 


period, Tp , and the complex amplitudes, c n • The effect of changes in the 
sampling parameters can be studied by using the mixing formula. 

More research needs to be done to further develop the mixing formula 
into an analytical tool. Since it relates the estimated coefficients to 
the "true" coefficients, it seems possible that an algorithm can be de- 
veloped that will improve the spectral estimate by "unmixing" the mixing 
effect. This procedure would involve an optimal search for the Fourier 
period by some appropriate criterion placed on the resulting spectrum. 

Such a routine has not been worked out by the author but some pre- 
liminary ideas were developed. These ideas are generally expressed in 
Hilbert space notation and employ some assumptions that have not been 


shown to be val id. 
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The basic problem will now be formulated. Suppose an observed x(t) 
is recorded for a time but it is assumed that the Fourier period is 
unknown. Using Hilbert space representations, suppose that the "direction" 
of the vector is changed until some "optimum" direction is found that gives 
as "best" estimate of the time series. This analysis would produce a 
period T Q p t which would approximate the Fourier period and complex coef- 
ficients (o' ,f) based upon the desired criteria. 

The state vector M the system is defined as the infinite series: 


® CD .)_» 

(x,T ) = £. f (x,T ) =2, 2 (o ,£)or (x,T J 


m m=l m m=l 


n n 


( 11 . 21 ) 


It wili be assumed that an operator, A, exists which will yield an 
"optimum" time series when operating on iji. A second assumption, based 
upon intuition and the need for a suitable criterion of goodness, is that 
the "optimum" period is the one which concentrates the spectral power into 
the fewest possible spectral components. This a priori assumption is, in 
effect, forcing the estimate to agree with the empirical data by approx- 
imating the data with the "least complicated" spectrum. To formulate this 
idea, we normalize the complex amplitudes such that the vector norm is 
unity: 


+® t 

£ ICJ =1 


( 11 . 22 ) 


The entropy measure for the spectrum will be defined as: 


H =- 2 Icnlloglcnl 
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To produce the fewest possible spectral components, we will want to min- 
i mize the entropy. H. The problem can be summarized as trying to find an 
operator, A, which operates on to yield an "optimum" time series, 
fUJopt)' which is the best estimate of the observed time series. The 
constraint for this optimization is that the entropy is minimized. Time 
did not permit the author to develop the details of this analysis or even 
to prove the basic assumptions. For the reader interested in developing 
this idea further, the author recommends using a calculus of variations 
with constraints. One example of this type of procedure has been given in 
Appendix i. Part J . 

Examples of the use of the spectral mixing formula and a further 
discussion of the use of the digital Fourier transform for the spectral 
analysis of finite time series will be presented in Chapter XII. 
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XII. FOURIER TRANSFORM SPECTRAL ESTIMATION 

A. I nfinite Periodic Time Functions 

A sample record of an infinite periodic time function with period Tp 
is shown in Figure 1 2 A - 1 . It is assumed that the function is bandlimited 
and that it can be characterized by 2M + 1 complex amplitudes. 



o 


Figure I Z A - 1 . A Sample Record of an Infinite 

Periodic Time Function with Period Tp 


This bandlimited function can be exactly specified for all time by a 
complex Fourier series 

+M +j nt 

x(t) = S c e F -os£t^+» (IZA.I) 

n-M n 


with complex amplitudes: 


1 +T f /2 -j ^ nt 

c =- f x(t)e F dt (IZA.2) 

n T F ^Tp/2 

The following quantities are defined: 
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Tp =The period of x(t) referred to as the Fourier period, 
u^, ' the Fourier frequency, also the lowest frequency com- 

ponent i n x(t). 

c Q =The average value of x(t). 

c n =The complex spectral amplitude or Fourier coefficient cor- 
responding to the nth frequency component, nu^,. 

=The highest frequency component in x(t). 

The actual number of complex amplitudes that are needed to describe a 
particular x(t) can be very deceiving. In a practical situation it may 
take hundreds of c n 's to characterize x(t) but many times only a few of 
them are nonzero. 


1. Fourier analysis 

The Fourier analysis of an infinite periodic function consists of 
determining the complex amplitudes, c n , and the amplitude spectrum, X(w). 
The complex amplitudes are determined from Equation I2A.2 and the ampli- 
tude spectral function is the Fourier transform of x(t): 




A % f 


(12A.3) 


The expression in brackets is recognized as a form of the Dirac delta 
function (Appendix II, Part E) and the final result is: 

-fM 

x(t)} = X(w) = £ C n 2iT 6(nau-cu) 

n=-M n ^ 



(12A.4) 
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The use of a delta function allows for the existence and definition of 
the amplitude spectrum of an infinite periodic time function. 

2. A utocorrelation function 

By applying the definition given in Appendix I for a time autocor- 
relation function and using Equation 12.1 for x(t), the time autocorrela- 
tion function for an infinite periodic time function is easily determined 
to be: 

+M ,2 +jn(jLL T 

R (T) = S Ic I e * (IZA.Sa) 

x n^M n 

i -2 M 

= le T +2, S, I ckl 2cos( kU I.T) (IZA.Sb) 

o k=l t 

R x (T ) has an average value of lc Q 1 and is periodic with period Tp. 

3. Power spectral density function 

The power spectral density function for an infinite periodic time 
function is determined by taking the Fourier transform of the time auto- 
correlation function: 

p{R (t)} = S (cu) = s IC. I 2n6(nuL-w) (IZA.G) 

J x x n -M n ^ 

For this ideal condition, the amplitude spectrum and power spectrum both 

A 

consist of 2M+1 line components of amplitude c n and Ic | respectively. 
They both give essentially the same spectral information about x(t). 

4. T otal spectral power 

The total power in the spectrum of x(t) is obtained by integrating 
the power spectral density function over all frequency: 
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Total Power = f s (w)dw = 2 |c l 2 (12A.7) 

^ atl w x n=- M n 

The total power is also obtained by evaluating the time autocorrelation 
function at zero lag: 

m 2 

Total Pover = R (0) = 2, lc I (12A.8) 

x n=-lv| n 

50 V ector norm 

The vector norm is used to determine the mean-square voltage or 
"average" power in the periodic time function. It is also a particular 
form of Parseval's Theorem. For the complex Fourier series it is given by 

1 +V 2 * -HI 2 

Average Power = — [_ x (t)x(t)dt = 2 lc I (12A.9) 

t F T f /2 n-M n 

From this representation we see that the total power in the frequency 
spectrum is determined by the mean-square value of x(t) averaged over one 
Fourier period. 


6. I deal sampling 

If x(t) is sampled by an ideal sampler and at a rate exceeding the 
Nyquist rate (see Chapter VI), the sampled function can be exactly recon- 
structed using the folloving infini.te orthonormal series: 


r si n rr(t - kAt) 
X ( t) = 2 x(kAt) [ „ 3 - ] 




n 


7^<t - kAt) 


I2A.IO) 


This formula can be used for exact interpolation between sampled values 
provided the sampling interval is less than the Nyquist sampling interval, 
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B. Finite Periodic Time Functions 

The infinite periodic time function x(t) is now observed for a finite 
time, T., , as illustrated in Figure 1 2 B - 1 . It will be assumed that the 
observer is unaware of the exact value of the Fourier period, Tp • Although 



x(t) 

, T 


/ \ 

\ 

\ ✓ 

'“'l^ 

L 1 _ 

t f 

1 I | I | | 

* \ 

! V/ 

1 \ \ 'am 

1 1 0 

| 1 




t 

0 

Figure 12B-I. 
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O 

Truncated Time Sample of an 

+ t n 


Infinite Periodic Time Function 


x(t) is infinite, a truncated time function can be defined using a rec- 
tangular observation window function such that 


x ( t) =u(t)x(t) 


where 


u(t) 


1 

0 


t +T, 

o o f 

otherwise 


or: 


x(t) 


x(t) 

0 


t Q s t £t 0 +T n 
otherwise 


(I2B.I) 


(12B.2) 


(12B.3) 
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For purposes of presentation, the origin in Figure I2B-I will be shifted 
so that to = “Tjj/ 2 and the sample length is symmetrical about the vertical 
axis. 


1. F ourier analysis 

Since the truncated function is absolutely integrable, 




lx(t) Idt 


. ^ 2 , 


lx(t) ldt< £ 


-v 2 


(12B.4) 


the amplitude spectral function exists and is obtained by taking the 
following Fourier transform: 


* 


( W ) =r[x(t) }=S J X(t)e" jat dt 

~-Y 2 


(12B.5) 


If x(t) in the above integral is replaced by Equation I2A.I and the inte- 
gration performed; the resulting estimated amplitude spectral density 
becomes: 


tf(w) 


•+M sin(nu^. - 

n i M cnT N[ (i^-W)T n 12] 


(12B.6) 


As the observation interval becomes very large, TN - ®, this estimate con- 
verges to the "true" amplitude spectral density given in Equation I2A.4: 


lim &u)> 


J -*CD 

N 


+M 

= 2 C n 2rr 5(noL. - w) 
n=-M n ^ 


(12B.7) 


The interested reader can verify this limit by applying Equation A2.49 
given in A ppendix 1 1 . 



132 


For the very special circumstances where the period of observation, 
TN 1 is an exact multiple of the Fourier period, TN = qTp , and if we are 
interested only in the amplitude spectrum sampled at multiples of the 
Fourier frequency, oUp =2TT/Tp , we get: 

-M . 

^< ku P "J, c n qT F < 12B - 8 ) 


The sine function in the brackets is actually a special form of the 
Kronecker delta function given in Appendix II. Replacing the sine function 
with the Kronecker delta and using the contraction property gives: 




' ^ S M C n 6 ,* = <V=k 

n=-M 


(12B.9) 


From this derivation we can see that, for these special circumstances, 
the complex spectral amplitudes can be exactly computed from the samples 
of x(w) by: 

c n =-’p ^(nu^) (12B.IO) 

This special relationship between the complex amplitudes, c n , and sample 
values of the estimated amplitude spectral function, )f(w), suggests a 
curious interpretation for E<?uation I2B.6. Except for a scale factor, T^, 
this equation is identical in form to the reconstruction equation (6D.6) 
discussed in Chapter VI. From this it is concluded that the amplitude 
spectral function for a truncated continuous time function can be obtained 
by a series expansion involving the "true" complex coefficients and an 
appropriate orthomormal sine function. 
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For a periodic time function, the amplitude spectral function can 
also be computed by applying the convolution theorem: 

(t) = U(t) X (t) (12B .1 la) 

^(u>) =U(w) *X(w) (I2B .lib) 


The Fourier transform of the rectangular window function is a sine function 
and x (w) is given by Equation 12A.4. The convolution of the sine function 
with a series of Dirac delta functions gives the same series as Equation 
12B.6. This convolution technique cannot generally be applied if x(t) has 
any random components because X(w) may not exist. The approach to use for 
random functions is discussed in Part C of this chapter. 


2. Estimated autocorrelation function 

There are two possible definitions for the estimated time autocorrela- 
tion function, 1$ (T). 'The first definition is valid for lag values much 
smaller than the observation period, t«t n < This definition is consis- 
tent with the usual assumptions applied in the B lackman-Tukey method: 

, -PL/2 * 

R(T)-i_r N x (t)x(t+T)dt t«t (12B.12) 

x ’S -v 2 


This definition also yields the simplest results 

+j u^kT 


+m -m 

R (T) = E I 
X 


k=-M L*-M ^ 


-Hi +j UL,kT 

= s Bje 
k=-M 


(12B.13) 

(12B.14) 


where 
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sinTT (k- L)^r 

_ * r _F 

Q kL k C L ^ T 

TT(k-L)^ 

F 

and 


(I2B.IS) 


a = E a (I2B.I6) 

K l^m ^ 

The amplitude constants, in Equation 1 2 B .14 play the same role as the 
amplitude constants, lc n "P, in Equation I2A.5. In this sense, the auto- 
correlation function for the truncated time function is no more complicated 
than that for the infinite case. The very definite difference between the 
two is that the estimated autocorrelation is itself a random function. As 
the window function moves with respect to x(t), it gives different sample 
realizations and thus generates a randomness in ^ x (t). This randomness is 
a direct result of the observation and sampling process. 

If the period of observation is again an exact multiple of the Fourier 
period (as in Equation I2B.8), the sine function reduces to a Kronecker 
delta and the complex amplitudes reduce to 


Q kL C k°L 6 kL 


(I2B.I7) 



The net result is that the estimated autocorrelation function is identical 
to the "true" function for these sampling conditions: 
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A J “V ‘ 

R x (t) = S 1^1 e = R x (t) 

k=-M 


(12B.19) 


The second definition for the estimated time autocorrelation function 
is -the one which will converge to the "true" function and hence is not 
biased. It is also valid for all lag values, |t|<TN. This estimate is 
more complicated than the previous one and does not lend itself to an 
easy interpretation. The estimate we will work with is given by: 


A 1 ^ (T N“ M>* 

= T - M I , , x (t " T/2)x(t+T/2)dt 
X T N It!) 


(12B.20) 


This autocorrelation estimate can be expressed in terms of the complex 
Fourier coefficients by replacing the x(t)'s with Equation I2A.I and doing 
the integration. After many detailed algebraic manipulations we obtain 


-tM ~ +jui,r 
R (t) = Z |c | e 


+m +m 


*■ S S e/c q (r)e 
k=-M L=-M L 


+ju^,(k + L) t/2 


(12B.2I) 


where: 


qkL (t) = 


1 Kl) +j ^ (L . k)t 

4- H j . . e c 

n -%ct n - M) 

sin (? Kl)] 

i V L ’ k) (Tjj - ItJ) 


I2B.22) 


I2B.23) 


The function qkL(T) is even and represented by two sine functions which are 
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mirror images of each other. Attempts at further analysis of qkL(T) have 
not been successful. The Fourier transform of q(r) involves a sine inte- 
gral. For Jt|«TN' q k L ( t) reduces to the sine function in Equation 
1 2 B .15 and the two estimates agree. 


3. E sti mated power spectral density function 

For the autocorrelation estimate given by Equation 1 2 B .12, the es- 
timated power spectral density function is 


£ x (oj ^ B n 2TTS(nUL - w) 

(t«T n ) n=-H 

where 

+M * 

3=2 ckc 
n k=-M n 


r TN 

I si n rr (k - n)^ 

L_ Tl , 

Tl(k - - 


(12B.24) 


(12B.25) 


and again if is an exact multiple of Tp , the amplitude coefficient 
simplifies to £ n = Ic f • 

For the autocorrelation estimate given by Equation 1 2 B .21, the power 
spectral density function involves the Fourier transform of qkL(T) which 
has not been determined. This estimate may be written as: 




O) 


-hi -m 
+ S E 
k=-M L=-M 
k^L 


* fr +J«t<k + WT/2 

Vl,? { W T > e 3 


(12B.26) 
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4. T otal power esti mate 

The total spectral power for each estimate is obtained by integrating 
the spectral density over all frequency. This gives for ITI «T^, 

1 *> A -*M 

Total Power = 21T U Sx(w)dw = £ 3 (12B.27) 

-00 n=-M a 


and for all |t j <1^: 


m 2 

T otal Power = I! Ic I 
n=-M n 


4M -+M * 


+ 

k= 


£ E ckc 
-M L=-M 


L 2 f: 



(T)e 


+j u^(k + L) T/2 


}dw 


k?^L 


(12B.28) 


It can be shown that the sum in Equation I2B.27 is positive and real. 


5. V ector norm 

The vector norm or Parseval's Theorem for the truncated function is 
similar in form to that for the infinite periodic time function. The 
major differences are that the mean-square value is averaged over the ob- 
servation interval and not over the Fourier period and the amplitudes, 

IC | , are replaced by the amplitudes, The approximation to Parseval.s 
Theorem becomes: 


i iV* 

J x 


(t)x(t)dt 


-Hi 

= S 8 
n=-M 



(uj)d(ju 


(12B.29) 


6. F requencv resol uti on and spectral mi.’ri.ag 

The ability to resolve line spectra and avoid false spectral peaks is 
of great concern when doing high resolution spectral analysis of periodic 
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signals. These topics can be studied by assuming a reasonable model for 
the spectrum of a periodic signal and then doing Fourier spectral analysis 
on the model. A convenient spectral model is shown in Figure 12B-2. The 

X (w) | 


Figure I2B-2. A Convenient Spectral Model 
for Resolution Studies 

periodic time function can be modeled by a Fourier series of four terms: 
x(t) = Ag COS (2TTfgt +0 a ) + COS (2rrf^t + 

+ cos (2TTf 1 t + 0^) + A 2 cos (2-rrf 2 t + 0 2 ) (12B.30) 

A complete Fourier series representation would involve determining integers, 
m, and a Fourier period Tp such that: 


f a =m a-T F = 

^ = “b f 2 =m 2T^ 


(12B.31) 
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The time function would then be modeled by a Fourier series such as in 
Equation I2A.I where many of the complex amplitudes would be zero. 

To be able to compute a Fourier series for the periodic waveform 
requires a knowledge of the Fourier period, Tp* If this is not known, it 
is difficult to determine the integers nr^, n^, and n^. 

The time autocorrelation function for an infinite record of x(t) is 
computed using Equation I2A.5 as: 

R x (t) = cos (2-nf a T) + ^ cos (2-nfj^T) 

+ I ^ cos (2T7f 1 T) + 2 A 2 cos (2-nf 2 T) (12B.32) 

If the infinite autocorrelation function were now truncated it would be a 
straightforward matter to use a lag window function and continue the 
analysis. The data function, however, is terminated with a data window 
and this complicates the analysis considerably. The estimated autocor- 
relation function should be computed using Equation 1 2 B .21. 

The complicated mathematical analysis of the estimated autocorrelation 
function will not be presented. Even the simple example of a single sine- 
wave burst discussed in Part 7 of this section involves a considerable 
amount of mathematical manipulation. Further spectral analysis for this 
section will be done using the amplitude spectral function. 

The amplitude spectral function for the infinite time function is 
computed using Equation I2A.4. For the four term series this becomes: 

X (w) =tt A g [e + ^ a 6(ai a + w) + e"^ a 6(a^ - w) ] 

+ tt A b [e + ^6(u^ ) + U)) + e ^^5(11^- U>)] 
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+ n A 1 [e + ^ 1 6(UJ L + 0)) +e"^6(cu 1 - U)) ] 

+ rr A 2 [e +j02 6(u^ + OJ) + e _j02 5(u^- tu) ] (I2B.33) 

We can say that infinite spectral resolution has been achieved in Equation 
I2B.33 because of the Dirac delta functions. For a finite observation 
period, the spectral resolution becomes finite. 

For mathematical convenience, the finite time function can be thought 
of as the infinite time function being observed through a finite data 
window. We write the observed function as the product of a rectangular 
window function and the infinite time function as: 

x ( t) = u ( t) x ( t) (12B.34) 


The rectangular window function was defined in Equations I2B.2 and I2B.II*. 
The amplitude spectral function for x(t) can be obtained from the convolu- 
tion of the Fourier transforms of u(t) and x(t). The relationship is: 


a 1 

x(w) =2JI I U(<a-X)X(X)dX 


(I2B.35) 


The Fourier transform of a rectangular window function with unity height 
and width i s: 


U(W) =T ll C 


sin u) 


.i 


I2B.36) 


W- 

2 


The convolution integral is evaluated using (12B.33) and (I2B.36) to give: 
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TN 


TN 


X (w) = A [■ ( W +W 


si n (w a + W) T- sin ( w 
a z a 


(W - w) 


2 J 


T T 

si n (u ^+ m) j* si n (u ^ - w) - 

+ (u^ + oi) + f ui - uf> 2 J 


(u^ - w) 


V 


TN TN 

s in ( + w) — si n ( - <jj> 


+ <u) 


( - uu) J 


+ A 2 [ 


TN TN 

sin ( + w)~- si n (a^ - n^— 

(u^ + cu) + (ujj - w) J 


(12B.37) 


The phase factors have all been omitted for simplicity because we are only 
interested in the amplitude of the spectral function. 

The spectral estimate above becomes exact when the observation inter- 
val goes to infinity because the sine functions converge to Dirac delta 
functions. This is illustrated in Equation A 2.49 in Appendix II. A 
typical spectral term in the estimator is represented by: 


A 


6 







(12B.38) 


The "spread" or width of this spectral component is characterized by the 
width between first nulls of the sine function, 


bandwidth-betveen-first-nulls 


= M iu 


4TT 


(12:8.39a) 


or by the bandwidth between the points where the amplitude is down by a 
factor of (2)^: 


BW 3dB 


16 tt 

9t n 


(12:8.390) 
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If we were to do a Fourier series analysis of x(t) and we knew the 
exact Fourier period we could set the observation interval equal to Tp 
and obtain an exact answer. This criterion of observation can be applied 
to the Fourier transform to determine the spectral resolution. If TN =1^, 
and we examine the typical frequency component at uu , the period of obser- 

a 

vation can be replaced by 


m 

^ =I f 

a 

where n^ is one of the integers discussed for Equation 1 2 B .31. 
is substituted into Equation L2B.37, the result becomes: 




m 


si n (2mf a - 2rrf) g 


m. 


(2TTf a -2TTf) Yf 



(12B.40) 
When this 


(I2B.4I) 


The ratio m./f. is multiplied through and, the sin term reduced to give: 

a a 


£ . 


Aa 


kn en»> a a--)3 


2 "< f - f a> 


I2B .42) 


The amplitude of the typical spectral component is evaluated by 
taking the limit: 


m 


liM 6 - A a f = A T - A a T H 
f-*f. a 


(I2B .43) 


The bandwidth-between-first-nulls is 




2w„ 




2f_ 


or 


Af FN = 


(12B.44) 


and the 3dB bandwidth is: 
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8w 

BW 3dB = ST 

d 

At this point a resolution quality factor or "Q" 
f m 

0 a = —2 
a 2 


(12B.45) 
could be defined as: 
(12B.46) 


We are now able to draw several conclusions from this example. First, 
if the Fourier period is exactly known, the Fourier series technique gives 
infinite spectral resolution. Second, the width of a typical spectral 
component in the Fourier transform is directly related to the frequency, 
f a , of the spectral component and the number of cycles, n^, that particular 
component has been observed. The spectral bandwidth in percent of the 
observed frequency is directly related to m a , the number of cycles observed. 
Third, for a given observation period, the higher frequencies will get the 
better percentage resolution because more cycles have been observed. This 
results in a higher ”Q”. 

The spectral resolution for the frequency components f^ and f ^ can be 
determined by examining the amplitude spectrum near these frequencies: 


x*(w) =* A . 


+A 


2 


TN 

sin (uij - w) - 

[ (0^- w) 2 J 

T 

sin - W ) y" 

^ (o^ - uu) ] 


(12B.47) 


Since and do, are nearly equal, the two spectral components have approx- 
imately the same spectral bandwidth. For equal amplitude spectra we de- 
fine the Rayleigh resol uti on as one-half the bandwidth-between-first-uulls: 
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X 2 TT 

Rayleigh resolution =r Auu =~ (I2B.48) 

N TJ 

Using this criterion, the two spectral peaks are just resolved. For the 
case where the observation period, T^, is equal to the Fourier period, Tp , 
the Rayleigh resolution is: 




(12B.49) 


Again we see that the more cycles observed, the better the resolution. 

We will also define a Fourier resol uti on as equal to the bandwidth-between- 
f i rst-nul I s: 


Fourier resolution • Aul_. - (12B.50) 

t N 

For this spectral separation we say that the spectral components are com- 
pletely resolved. 

The resolution criterion must take into account differences in spectral 
amplitudes because a large component near a small one may obscure the 
identity of the smaller component. A criterion is difficult to establish 
'for dissimilar amplitudes. The "leakage" of spectral power from the large 
component onto the frequency of a smaller component can be fairly well-de- 
fined. For a Fourier resolution criterion we will say that a component 
has been resolved if the estimated spectral amplitude is larger than twice 
the amplitude of the leakage spectra. In terms of the example, this means 
that: 

TN 

>2A [ Sin(4l W2, 

1 



(I2B .51) 
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The criteria used to select an appropriate observation interval will 
now be summarized. First, to resolve the lowest frequency component, f . 

a 

the number of cycles to be observed depends upon the absolute frequency 
and the desired bandwidth between first nulls: 


# 


of cycles observed =rr^ = 



(I2B.52) 


The needed observation length for distinguishing the lowest spectral com- 
ponent from zero frequency is obtained by equating one-half of the band- 
width-between-first-nulls to f g . This gives, n^ =4. Second, to resolve 
two closely spaced sinewaves by the Fourier resolution criterion requires 
an observation period of: 




I2B.53) 


FN 


Third, to determine the observation period required to resolve the band- 
width, f^- fa' into rr^ increments of frequency resolution it is necessary 
to compute the bandwidth-betveen-first-rrulls as the ratio of the total 
bandwidth to the total number of increments. This is used in (12B.53) to 
give: 

2m_ 

(12B.54) 


2m , 

T N > r-f 


s 

V a 


7. Sinewave burst spectral analysis 

In somewhat of a digression from the material in the previous section, 
we consider a single frequency sinewave .observed through a data window of 
width, t^. The phase of the sinewave with respect to the origin of the 
data window is arbitrary and contributes a random aspect to the sampling 
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observation. The observed function illustrated in Figure 12B-3 is some- 
times referred to as a sinevave burst. The infinite cosinusoid is ex- 
pressed in the phase angle representation as 

x ( t) =A cos (w Q t + 0 o ) (12B.55) 

with amplitude spectrum, time autocorrelation function, and power spectrum 
respectively: 

X(uq) = tt A[e + ^O6( 0Do + uu ) + e" j0 o5(uj o - m)] (12B.56) 

R x (t) = -- A 2 e° s in o T (12B.57) 

P x (w) =g A 2 [5(w 0 +w) + 6(w Q - w) ] (12B.58) 



Figure 12B-3. Sinewave Burst or Single-Frequency 
Sinewave Observed Through a Data 
Window 
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For the Fourier spectral analysis of a sinewave burst of arbitrary 
reference phase, we can model the observed function as the product of a 
rectangular window function and the infinite time function: 


x{t) =u(t)x(t) =u{t)A cos (w Q t +0 q ) 


(12B.59) 


The rectangular window function u(t) was defined in Equations 12B.2 and 
1 2 B .11. The amplitude spectrum of x{t) is obtained from the convolution 
of the Fourier transforms of u{t) and x{t). The relationship is: 


x*{w) 


= •— do{w- X)X(X)dX 
Ztt rn 


The Fourier transform of the rectangular window function is 

sin (Dr- 

U ( W ) =Tu 0 ^ ] 


(I2B.60] 


(12B.61) 


and the convolution integral becomes: 


i 4 sin (w +w)r- ,, sin (w n - 

Am) =A[e + ^o „ + e .St a - —2 — — ] (12B.62) 


(W 0 -FW) 


(w 0 - w) 


This is the familiar amplitude spectral function previously presented in 
Equation 12B.37. The sinevave burst may be effectively analyzed with this 
function. If one applies the autocorrelation concept to the sinewave burst 
in an attempt to obtain the power spectral density function, the mathe- 
matics becomes very complicated. 

For the sinavave burst, the estimated time autocorrelation function 
can be computed using either the definition in Equation 12B.20 and inte- 
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grating directly or by applying Equation 12B.21. For this discussion we 
will apply Equation 12B.21 to demonstrate its usefulness to the reader. 

In complex notation, the sinewave is written as 

x ( t) =c + c+ e +JU *> t (12B.63) 

where (see Appendix 1 1) : 

c =j A e-J'^o c + = '2 A e+j0 ° (12B.64) 

A pplying (12B ,21) we get 


with: 


k (t) = le |V ju b T + lc | 2 e + j»*> T 

X - 1 +' 

+ c*C +q-+(T) + c*c_ q+ _(T) 


(12B.65) 


q.+C'O = q+.CT) 


SinU, o (T N~ M> 

"WW" 


(12B.66) 


Simplifying terms and adding we can obtain the estimated time autocor- 
relation function for the sinewave burst as a function of both lag and 
observation period 


i< T 'V =1 A 2 COS U> o T 

1 2 sin w o (JN - M) 

+ 2 A C0S 2 ^ o L [rfi ] 


(12B.67) 


where it must always be remembered that the lag must be less than the 
observation period, and that the autocorrelation is zero for lags 

larger than T^. 
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r^(t,Tjj) is really a function of the random variable, introduced by 
the randonmess of observation. The expectation value of the estimate is: 

e{r x (tJN)} =r x (t) M<T n (I2B.68) 


The last term in (12B.67) is the direct result of not having an infinite 
record to average over time and thus represents a distortion in the auto- 
correlation function obtained for an infinite sinewave. We now examine 
r ( t,t„) to determine the effect and seriousness of the distortion. 

The two terms of the sum in (12B.67) are plotted in Figure I2B-4. 

The most serious distortion of the ideal cosinusoidal shape of the auto- 
correlation function occurs within a distance, T Q , of the observation 
interval, TN. For small observation intervals, the sine function will 
"mask" the identity of the sinusoid. For large observation intervals, 
the effect of the added sine function is reduced to a small percentage. 

We will look at two limiting conditions for ^ (t,t ) and compute the 
limiting effect on the amplitude estimate and the spectral estimate. For 
lag values much smaller than the observation interval and for an observa- 
tion interval much larger than the period of the cosinusoid: 




X (T ’V a2cos V = R X (T) 


T «T, 


N 


(12B.69) 


T » T 
N o 


This result is expected because we want the estimate to converge and be 
unbiased for large observation intervals. Next we look at the estimate 
of the total power because it is used to obtain the estimate for the 
mean-square value of the cosinusoid which is in turn used to estimate the 
amplitude, A. For zero lag, the total power estimate is: 





Figure 12£-4~ Two Com 
A utocor 
B urst 
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sin ci) T 

Total Power = R (0 ,t) = iA 2 [l + cos20 —4 (12B.70) 

x N 2 o OJ T„ 

o N 

A plot of this function is shown in Figure I2B-5 to illustrate how the 
total power estimate converges as the observation interval increases. 



0 




5T 

2 


o 


Figure I2B-5. Convergence Function for the Total 
Power Estimate of a Sinewave Burst 


The maximum error in total power is 2 or a factor of 3db. The absolute 
percentage error for worst case phase and frequency is obtained by 
evaluating: 


±a 2 [i± 


0) T 
o N 


] 


(12B.71) 
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Table 12B-I shows the observation interval as Some multiple of the_period 
T q and the resulting percentage error in total power. It must be re- 
membered that these are absolute worst case values and actual results may 
be considerably better for observation periods exceeding T. Also, the 
randomness of the initial phase of the cosinusoid will have the effect of 
producing a randomness in the estimate. 


Table 12B-I. Percentage A mplitude Error as a Function of Observation Inter- 
val 


Observation 1 nterval 

Percentage E rror 

^ - i -»*o 

± 10% 

I, = 3.181 

N O 

± 5% 

T = 15.9 T 
N o 

± 1% 

T. t = 159 T 
N o 

± 0.1% 


The Fourier transform of R x (t,t n ) is used to obtain an estimate of the 
power spectral density function for the sinewave burst. The power spec- 
trum can be represented by two components 


- s^uj) +r 2 (u>) 


(12B.72) 


w 


here the integrals over the interval ± become: 
§>) =|-A 2 J. cos w 0 Te" ja/T dT 


(U B .73) 
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S2(UJ ) = '2 A 2 cos 20 q 


+TN sin UJ (TN - 

S — a — 

-T 

N 


!) 




-jure, 

e d.T 


(12B.74) 


The analysis could be continued using these definitions but much more in- 
sight into the processing problem can be gained by introducing a lag 
window function, a plot of the estimated autocorrelation function 

showing the application of a rectangular window function is shown in Figure 
I2B-6. The estimated power spectral density function becomes 


where: 


S* x (tu) = ^(uu) + ^ 2 (uu) 


S x (uj) 


A 2 S coswi-e-j^dr 


-T 


(12B.75) 


(12B.76) 


£>(u>) 


1 A 2 O* 

2 A cos 2 0 q 


+T. 

I 


L sinu) o (T N - |t|) 


cu 


— T 


tvW 


JWT 


dT 


(I2B.77) 


These estimates will converge to those in (12B.73) and ( 1 2 B ,74) when 

Since both parts of r x (t,t n ) are even, the cosine transform can 
be applied to give: 


1 2 _ sinew - W)T L sinew +W)T L 


(12B.78) 


S f 2 (03) 


+t l 

iA 2 cos2« o ; 

- T L 


sin ""o (T N* hi) 

®o<V 


COS UJTdT 


(12B.79) 
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Figure I2B-6. A Rectangular Lag Window Function Applied to 
Estimated Autocorrelation Function of a Sine- 
wave B urst. 
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These two spectral components must be analyzed to determine the accuracy 
of the estimate. The !)l(w) component is exactly what would be obtained 
from applying a rectangular window function to an infinite autocorrelation 
function obtained from an infinite time function. In this sense, £l(w) 
represents the "true" spectrum obtained by a finite observation. The 
5* 2 (W) component represents the additional spectrum generated because the 
time function was truncated before the autocorrelation function was 
computed. 

The expression for !d 2(W) given in (12B.79) could not be integrated 
directly because an analysis of the integral by a change of variables 
showed that it was a sine integral and could only be solved by numerical 
techniques. If it is assumed that the lag window width is much smaller 
than the observation interval, T L <<1^, the spectrum represented by s* 2 (ui) 
can be written as 


a 12 sinu) o T N o L 

S 2 (w ) = 2 A cos 20 q — - — C> cos urrdT 

X L <<lk ° - T L 

2 s]_nvwIJi sLqwIl J 

=at l cos 2 V in b t 


(12B.80) 

(12B.8I) 


The power spectral density function for this special case is plotted in 
Figure 12B-7. The effect of truncated data and a rectangular data window 
on the sinewave burst power spectrum is an added sine function term at 
zero frequency and a distortion of the total power estimation. The ampli- 
tude of the dc term is a factor of 

si nw T 

2 cos 2 0 O C J 


(I2B.82) 



S x (w) =S 1 (uu)+^ 2 ( U j) 


/ — A T t cos 2 0 [- 


sin id T 


Figure 12B-7. Power Spectral Density Function for a 
Sinewave Burst where «TN 


smaller than the "true spectral amplitude of w) • For practical sit- 
uations, T n »T o> and the worst case value for this factor becomes: 




(12B.8 


For a dc amplitude of less than one-percent of the "true" amplitude, the 
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sinewave burst must be observed for more than 31.8 cycles. 

These derivations for a simple time function such as a sinewave burst 
help illustrate the mathematical difficulty associated with trying to 
calculate the power spectrum for a truncated periodic function. These 
problems are usually solved numerically. 

8. E rrors in amplitude spectral esti mation 

A continuous but finite record of a noiseless periodic waveform with 
Fourier period Tp is sampled for a time TN. The accuracy of the Fourier 
coefficients calculated from this sample record will depend upon the length 
of the sample and the relative amplitude of each frequency component in the 
spectrum. The Fourier transform estimate of the amplitude spectrum is 
given by Equation I2B.5. This estimate can also be written in terms of 
the Fourier frequency, the "true" complex amplitudes, and the observation 
interval as given in Equation I2B.6. 

There are actually four stages of estimation which may be considered 
in determining the accuracy of our estimate. Stage-1 is represented by 
the actual Fourier transform given by Equations 12B.5 and 12B.6. These 
estimates are continuous functions and do not directly represent the com- 
plex amplitudes. Stage-2 is the discrete spectral estimate obtained from 
doing a discrete Fourier transform such as the FFT. The discrete spectrum 
obtained by this approach is represented by: 


,+T /2 -j 2fT kt 
X(k = J N x(t)e hi dt 

* -V 2 


(12B.84) 


or by modifyi ng (12B .6): 
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x(k |2) 


n=-M 
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2 ‘A 


sl.n( 


n21T k_2n 


F 




>V 2 


'f 2 - r^V 2 

F N 


(12B.85) 


This discrete spectrum mayor may not correspond to the spectral frequen- 
cies representing the "true" Fourier spectrum. If the Fourier frequency 
were known, a stage-3 estimate would be defined such that 


+T N /2 


&kup = J x ( t) 




dt 


or: 


^(kup = 


■v 2 


-H4 


S C T 
n=-M nN 


T 

si mi (n - k)~ 

~T 

T 

rr(n - k)r? 

F 


(12B.86) 


(12B.87) 


Equation 12B.87 can be solved to give an exact expression for the k-th 
complex amplitude as: 



-m 

2 c 
n=-M n 

n^k 


siniT (n-k)~ 

TT(n- k^ 


(12B.88) 


The second term on the right-hand side of (12B.88) can be thought of as an 
error correction factor for the Fourier transform estimator. If the 
Fourier transform approximations for the complex amplitudes in the error 
correction factor are substituted into (12B.88), the stage-4 complex 
amplitude estimator is obtained: 
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n=-M 

n^k 


X*(nu^) 


si nil (n- k)~ 
iF 

T ~ 

TT(n - 

F 


(12B.89) 


This estimator is not practical unless the Fourier frequency is known or 
can be accurately approximated. 

The unknown parameters for the estimation problem are Tp and c • 

These must be determined from x(t) by using t n and Equations I2B.5 or 
12B.84. The error associated with the amplitude spectral estimate is 
mostly dependent on the observation time, TN. Equation 1 2 B .10 showed that, 
if TN was an exact multiple of Tp, the complex amplituded could be computed 
exactly. When TN is not an exact multiple, we can use Equation I2B.87 to 
estimate the error. First let 


T = qT + AT (12B.90) 

N r 

where: q =1,2,3, ••• and 0£&T<T_. 

t 

This is substituted into I2B.87 and after many manipulations we obtain: 

+M 

^(koL.) = Z c q T F cos TT^n -k ) ^ [si nTT (^k)9] 

* n=-M n F TT n - q 

^ si n TT (n - k)^ 

(12B.9I) 


This can be further reduced to: 
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^(kctL.) = qT p E c cos Tr(n-k)^ [6 ] 

n=-M F 

-HI AT 

+ & S c cosTT(n-k)q siacC(n-k)^] (12B.92) 

n=-M n l F 

The first term on the right-hand side of (12B.92) is reduced to qTp by 
using the sifting property of the delta function. The second term is an 
error term caused by not sampling at an exact multiple of the Fourier 
period. The principal error term, is iTc^ and is found by letting n =k. 
The estimate of (12B.93) is now written as: 


= <qT p + AT)c k +AT 


+M 

E 

n=M 

n^k 


c cos TT(n - k)q si nc[(n - k#] 
n t f 

(12B.93) 


This equation reduces to Equation I2B.9 when &T-*0. As the number of 
periods of observation increases, the term qTp dominates and the estimate 
converges. 

C. F i nite Random Time F unctions 


A sample time function for a random process exists for all time, 
-®£t A truncated time function is defined from this sample by 

multiplying it with a rectangular window function such that 

=u T (t)x(t) (I2C.I) 

where u^t) was defined in Equation I2B.2. For this condition, ^(t) is 
absolutely integrable and an amplitude spectrum exists and is defined as: 
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£_(a>) =5W,(t)e-j u£ dt = S x(t)e" jat dt T<® (12C.2) 

1 -as 1 . -772 

One might expect to apply the convolution theorem to obtain ^(tu) but this 
requires the existence of the Fourier transform of the infinite time 
function, x(t). For random processes, this transform generally does not 
exi st. 

This conceptual dilemma can be resolved through the use of Parseval 's 
theorem. The power spectral density function sought is the distribution 
of average power over frequency so for the previous estimations this is: 


? J x£(t)dt = “ J I dw 


T < CD 


(12C.3) 


The term on the left is an estimate of the mean-square value of x^(t) 
averaged over a time, T. Both this estimate .and are random because 

they are derived from only one sample function of the process and hence 
are not ensemble averages. 

These estimates of average power can be used to derive the power 
spectral density estimate by first taking the expected value of both sides 
of (12C.3) and then taking the limit as T - ®: 


+T/2 


lim * ^ E{xf (t) }dt =f- J lim t e{ l 2 }dw (12C.4) 
T-»<=° -T/2 -® T-“ 

We know, fora stationary process, that the total power is determined by: 


E ( X 2 (t) } = (w)dw = yO) 

-.CD 

o 

The expectation of ^(t) can be reduced as follows: 


(12C.S) 
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E{x^(t)} = u^(t) E [ x 2 (t)} =u^(t)R x (0) (12C.6) 

This can be used to reduce the left-hand side of Equation I2C.4) and from 
identification with (12C.5) we can conclude that: 


s (uu) = limi e{|jL(o))| 2 } (12C.7) 

x X -* 00 x L 

This gives the power spectral density function in terms of the estimate 
of the amplitude spectrum. 

To show the relationship of this estimate to the autocorrelation 
function definition of spectral power, the Fourier transform relationship 
for x t ((d) is substituted into (12C.7) to give: 


1 | +T/2 + jiu*i | +T/2 *jiu*2 

S x (w) =lim-E{ J x(tl)e dt| J x(t 2 )e dt 2 } (12C.8) 


J -»os 


-T/2 


+ T/2 +T/2 

= lim - E[ J J 


ml_L W W x(t|)x(t-j)e 
T - ® -T/2 -T/2 


-T/2 


-juXt2 - 1 ^ 


dt, dt 2 } (12C.9) 


This double integral is further simplified by using the variable substitu- 

i, t| =t: 

+ 772 - t +T/2 


tion, T=t 2 ~t|, and then, t| =t: 


Sx(w) = lim 

T-® * -T/2- 


| J J 

* -T/2-t-T 


/ 2 


E [ X ( t) X ( t+, . ) }e _jUJT dtdT 


( i2c .io ; 


For a stationary process, the expected value inside the integral is the 
time autocorrelation function F^Ct) and since it is independent of reference 
time, t, the integral equation reduces tor 


i+T- t 


5 V (w) = lim 

A 


T - ® -T - t 


F^(T)e"j ltm dT 


= (T)e- jtOT dT 
•€0 


(12C.II) 
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Equation I2C.II shows that the estimate for power spectral density given 
by (12C.7) converges and is unbiased for large observation times. 

All of the estimation techniques which have been developed for ampli- 
tude spectral analysis can be modified and applied to finite random time 
functions by using Equation I2C.7. An estimator for power spectral density 
for an observation time T would be: 

Sx(w) = £ Etl^Cw) I 2 } (12C.12) 

D. F inite Periodic Time Series 

The previous analysis of periodic time functions and random time 
functions could be applied to discrete time functions by "digitizing" all 
of the continuous equations and functions. This approach can be taken 
when the collected data is continuous and the desired degree of numerical 
resolution may be chosen. This technique is equivalent to a numerical 
solution for the continuous case. 

Another approach to the analysis of discrete data is to treat the 
data directly as though it is a time series. The author prefers this 
approach because it results in the use of superior data processing tech- 
niques. Data in the time domain will be treated in discrete form even 
though it would be possible to generate a continuous function using the 
sampling theorem of Equation 6D.6. 

A periodic time function, x(t), is sampled for a finite observation 
time, T^, with a sampling interval, &t. This process will generate a 
time series x(k&t). The sampled function is shown in Figure I2D-I and 
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we observe the following definitions: 

At = The sampling interval. 
fs The sampling frequency. 

2N+I = The number of samples of x(t) in the interval TN. 

TN =The length of the observation interval. 

Tp =The Fourier period of x(t). 

2 n 

mul = m — , The highest frequency component in x(t). 

1 F 

f p =2 f s ' The folding frequency. 

2^ = 2Mo^, is the Nyquist frequency. 



Figure 1 2 D - 1 . A Sample Record of a Finite Periodic Time Series 


It will be assumed that x(t) is sampled at a rate greater than or equal to 
the Nyquist rate so that aliasing is prevented. Also, the number of 
samples will be assumed to be odd so that; (2N + l)&t = T . 
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1. F ourier analysis 

The discrete Fourier transform has been briefly discussed in Chapter 
XI and Appendix 1 1 . The discrete transform pair in the time domain can be 
represented by: 

. 2TT , 

+J - 

1 +N , 2TT T F 

x(kAt) =- Z X (n -T)e (12D.1) 

TF n=-N F 


X(n |2) 
F 


+N 

At E x(kAt)e 
n=-N 


-J" j TT nkAt 


(12D.2) 


If the periodic function is sampled such that, At=Tp/(2N +1), the discrete 
transform pair becomes 


X . = 
K. 


t L 

n=-N T F 


+ j* 2»1 
X e J 


nk 


(12D.3) 
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-j 


2TT 

2N +1 


nk 


2TT 

where the arguments kAt and n ~ have been replaced by the indices k and n. 

Most discussions of the discrete Fourier transform actually use the 
complex Fourier coefficients, c , as the amplitude spectral component when 
actually it should be X n . This is done in B erg land (1969) and in Equations 
11.9 and 11.10 of this text. As a practical matter it makes little dif- 
ference as they differ only by the scale factor Tp • This can be seen by 
comparing Equations 11.9 and 11.10 with 12D.3 and 12D.4. For a given time 
series k =0, ±1, ±2, •••,±N}, the complex amplitudes computed by 
(12D.4) represent those which are needed to exactly reproduce the original 
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time series using Equation I2D.3. This feature of the discrete Fourier 
transform is discussed in Chapter XI. In the discussions to follow, we 
will use the circumflex accent over x. and x„ to denote that they are 
really estimates and the exact Fourier period, Tp, will be replaced by the 
observation period, TN. Also, in most cases we will work with the complex 
Fourier amplitudes, c n =x n IT N * 

The theory behind the discrete Fourier janalysis of a finite time 
series must be studied to determine the interpretation of the estimated 
spectrum. Given a finite time series (x(kit): k = 0,+l,+2, we 

have 2N+I ordered data pairs {x^k}. This data set can be used to define 
2N + 1 complex Fourier coefficients according to the following algorithm: 


N 2tt nk 

a 1 + -J 2TH-I 

c =2N + 1 , £ V 5 
n k=-N K 


(12D.S) 


These complex coefficients are actually approximations to the "true" co- 
efficients representing x(t). These estimates are then used to defi ne a 
bandlimited Fourier series of the form: 

+j 2™ t 

a +N a TN 

x(t) = Z ce TN = (2N + 1) At (12D.6) 

n-N a 


This estimated Fourier series has the following properties: 

1. x(t) is continuous and periodic with period, TN =(2N + l)At. 

A 0 *-r 

2. The highest frequency component present in x(t) is, 2^^ ='n ~ = 

N N 

2n 2N + 1 f s ’ 

3. The average value of x(t) is c • 
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4. The mean-square value of x(t) is computed by: 

-> +N o 

x = I |c r (12D ,7a) 

n=-N 

5. The critical frequencies of the discrete transform are: 

f n = tN = 2I\P+1 f s ( n=±l,+2, > +N) (12D ,7b) 

The most important feature of x(t) is the fact that the values of x(t) at 
t = kAt are exactly equal to the data set values x(kAt) , in other words: 

x(kAt) = x(kAt) , k= 0, +1, +2, . . +N (12D.8) 

The complex spectral amplitudes in Equation I2D.5 represent the spectrum 
of x(t) exactly while x(t) approximates x(t). It seems reasonable to 
assume that if enough samples of x(t) are taken so that it is very closely 
approximated by x(t), the spectral estimate, c , will approximate the true 
spectrum, c n • 

The continuous function, x(t), generated from the * n by Equation 
120.6 and hence from the f i nite time series {x(kAt) : k= 0, i I , +2^ **» +N) 
can be said to be "optimum" in the sense that x(t) at t =kAt exactly 
reproduces the original time series with a Fourier series bandlimited to 
N/TNo This bandwidth limitation provides the "smoothest" extrapolation 
between data points consi stent w ith the restriction that x(t) be periodic 
with period, T„. The fact that x(t) is both periodic and bandlimited 

N 

provides considerable mathematical convenience for analysis purposes. 
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To illustrate the discrete Fourier transform we will look at a simple 
example. PROGRAM 01 in Appendix III has been written for the example. The 
line spectrum subroutine in this example is a straightforward digital 
transform. For large amounts of data processing, th$ FFT should be used. 
Consider a time series obtained by sampling a continuous time function 
represented by (see Appendix II) 

x ( t) =4 cos(2iTt) + 1 sin(6nt) (12D.9a) 

where, a 1 = 4 volts, f ^ = 1 HZ., b 3 = 1 volt, and f 2 = 3 Hz. The Fourier 
frequency is 1 Hz. and the highest frequency is 3 Hz. The complex ampli- 
tudes are calculated to be: 


co 1 c 2 =0 


* 



(I2D ,9b) 


The data inputs to the program are the a's and b's, the Fourier period, 
the period of observation, and the number of data points'. The program 
generates x(t) sampled at the intervals =TN ! (2N + 1) and calculates the 

magnitude and phase of c p for, n=0, 1, 2, •••N. The magnitude of with 
respect to a chosen reference is also given. Finally the calculated c* n 
are used to generate the estimated series x(t) where x(t) is sampled four 
times as often as x(t) . 

By varying and 2N + 1 it is possible to see the effect of sampling 
on the computed discrete spectrum for the time series {x(k&t); k=0, +1, 
+2, Figure I2D-2 shows the effect of observing for less than a 
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Figure I2D-2. Discrete Spectrum Plots Showing 

the Effect of a Varying Observation 
Period 
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Fourier period and also shows how the spectrum converges as approaches 

Tp • Several computer runs where TN was an exact multiple of Tp showed that 

the calculated spectrum was exact within computational accuracy. The 

magnitude of the complex amplitude was plotted in dB with a two volt 

reference. A dB plot is necessary because of the wide range of amplitude 

values. It is important to note that the plot for TN =0.7 shows no peaks. 

This may be described as a shift of spectral power toward dc. 

Figure I2D-3 shows the calculated spectrum for a variety of sampling 

conditions. It is important to note that the spectral resolution of 81 

data points and T = 10.5 is little better than the resolution of 21 data 
N 

points and 1^=3.!. This illustrates the very important effect of obtain- 
ing and observation period as close to a multiple of Tp as possible. The 
reader may experiment with these effects by using PROGRAM 01 to study a 
variety of sampled functions and sampling conditions. For'large data sets, 
a Fast Fourier Transform subroutine should be used in place of SUBROUTINE 
LNSPTM. The high degree of spectral selectivity observed as TN is varied 
was the reason for the discussion at the end of Chapter XI of the selection 
of an optimum observation period. This procedure has much potential in 
terms of improving the discrete Fourier transform. This theory needs to 
be worked out. 

2^ S pectral mixing 

A discussion of spectral mixing and the derivation of the spectral 
mixing formula were given in Chapter XI. In this section we discuss the 
use of the spectral mixing formula 
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Figure 120-3. Discrete Spectrum Plots Showing the 
Effects of Long Observation Times and 
Increased Number of Samples 
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A +M 

C = E c 6 (N,T T ) (12D.I0) 

q r =-M r rq N F 

where the estimated Kronecker delta function is defined by: 

6 “r q < N -VV - a«Tr cos 2^1 ( fr'T- )T N ] < 12DII > 

rC— 1 F N 

The estimated Kronecker delta has the following properties: 


r=q 


rc l [8 r#qandT N = T F 


; 1 2 D .12) 


£ rq ~0 r5tqandT N »T F 


(12D.13) 


The ranges on the indices are: 


r —0, ±1, +2, +M 

q =0, ±1, ±2, +N (12D.14) 

PROGRAM 02 was written to verify the formula for the estimated 
Kronecker delta function. RUN #1 shows all values for the condition 
TF=TN=I. and M=N = 5. Within a high degree of accuracy, Equation 
1 2 D .11 is verified for the conditions specified by Equation 1 2 D .12. For a 
large number of samples, the estimated Kroneker delta converges to the 
"true" delta function.- This can be verified using PROGRAM 02 by increasing 
N and TN. The -i nterested reader is encouraged to experiment with the 

A 

properties of by using the program. 

There are a variety of ways the spectral mixing formula can be applied 
to the analysis of the spectrum of a time series. A particular application 
is needed before a specific use can be specified. One of the most useful 
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applications is the determination of the amplitude resolution limit for 
an estimated spectrum. The determination of resolution limits will be 
discussed in Chapter XIX. 


3. Spectral resol ution and accuracy 

A discrete frequency spectrum derived from digitally processing a 
time series can be viewed as a sampled frequency spectrum. Because of 
sampling, the spectrum will have a limited resolution and accuracy and it 
will be finite. For a periodic time function, x(t), the maximum and min- 
imum frequencies present are: 


W = 211 M 

max '|p 


max 



ID 


min 


2\T 

'IF 


min 



(12D.15) 

(I2D.16) 


For a time series with sampling interval, At, and observation interval, 
T N =(2N + 1) At, the maximum and minimum frequency components detected by 
the digital transform are: 


U) 

max 



f = N L_ 

max 2N +1 At 


(I2D.I7) 


tu . 

min 


2tt 



(I2D.I8) 


The digital transform can be viewed as generating discrete frequency "boxes" 
or frequency bands into which the computed power in the sampled signal is 
distributed. A continuous spectral function is generated from the discrete 
spectrum by expanding in a series of sine functions (see Equation I2B.6): 
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The bandwidth between first nulls is: 


BWFN = 2N“+ 1 f s =-^ 


(12D.19) 


(12D.20) 


The center frequencies for the "boxes" of the discrete transform are 


f 

n 


n 1 
2N+1 At 


2N+ 1 s 


n 


(12D.2I) 


where f is the sampling frequency. A convenient technique for concep- 
tualizing this process is illustrated with the help of Figures I2D-4 and. 
I2D-5. 


4. Sampling and "sync" frequencies 

The selection of a sampling interval, and an observation period has 
a profound effect on the estimated spectrum when the total number of 
samples is limited. As we discussed in the Fourier analysis section, a 
value of which is exactly a multiple of the Fourier period will produce 
an exact spectrum up to a frequency of N/T^. The sampling interval, 

At =T N /(2N + 1), may determine how many unique samples of a particular 
sinusoidal component are obtained. Consider the effect of the sampling 
interval on a single sinevave as shown in Figure 12D-6. This is effec- 
tively the same as sampling a single period of the sinewave over many 
points as shown in Figure 12D-7. 




Figure 12D-4. Conceptual Presentation for Illustrating 
the Distribution of Spectral Power as the 
Number of Samples Increases 
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N = 12 

(25 Samples) 


f 


Figure 120-5. Conceptual Presentation for a 
Large Number of Samples 



This way of looking at the sampling process led to the discovery of 
unique sampling frequencies we will call "sync" frequencies. When the 
spectrum of x(t) contains a sinewave component corresponding to one of 
these sync frequencies, the number of unique samples of this spectral com- 
ponent is limited to a finite number, • 

To determine these frequencies, consider the single sinewave in Figure 
12D-8. We can define a remainder period AL such that: 

2 AL =T - n At 

P 


n= 3,4, 


( 120 . 22 ) 
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Figure I2D-6. Sampling a Single Sinewave 



Figure I2D-7. Equivalent Sampling Information 

for One Period of a Single Sinewave 
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2 At 

Figure I2D-8. Single Period Sampling of a Single Sinewave 
Showing Remainder Period, al 

The integer, n^, represents the number of increments in a period, T, before 
the cycle repeats and AL is one-half of the remaining period length. 

For, AL=0, the sync frequency and unique sampling interval are de- 
fined by: 

AL = 0 At =- 1 T f n = T 2 (12D.23) 

Ap Sn ' 

For a 3 Hz sinewave and, n =7 unique samples, the sync frequency would be 

P 

21 Hz and the sampling interval would be 1/21 seconds. It should be noted 
that any integer multiple of a sync frequency is still a sync frequency. 

For, AL >0, the total number of periods sampled must aCCUIDUIate until 
npAL=T. The sync frequency and unique sampling interval become: 

11 2n 1 

At= ( 2 - -)T f sn^n e - 2^~T 

P P 

n p =3,4,5,. "(2N + 1) 


AL >0 


(12D.24) 
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For the previous example, the sync frequency would be 42/5 Hz and the 
sampling interval would be 5/42 seconds. 

Another way to view sync frequencies is to select a given sampling 
frequency, f s ' and determine where in the spectrum these frequencies will 
occur. They will occur at frequencies corresponding to: 


f 



and f = 

i. n s 
P 


A L = 0 


AL >0 


(12D.25) 


The location of these frequencies with respect to the discrete spectrum 
is shown in Figure 12D-S for 3^n p ^10. This figure should be compared to 
those in Part 4 of this section. From the figure it can be seen that the 
frequency f g /3 has the fewest unique samples of all the sampled frequency 
components. For n p =3, the critical Fourier frequencies are 

1/7 f 5 . 2/7 f s . 3/7 f 5 

as shown in the figure. Only the first component corresponds to a sync 
frequency. 

As stated previously, the total number of independent samples for any 
given sync frequency is n • It does not improve resolution to sample for 
an observation interval longer than, TN =n p At. There is an exception to 
this however if the sinewave is corrupted by noise. With a signal that 
has additive noise, the resolution accuracy can be improved by increasing 
the observation period because this will provide an averaging effect for 
each of the unique sample values defined for the noiseless case. 
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Figure I2D-9. The Locations of the Principle Sync 
Frequencies as Referenced to the 
Sampling Frequency, f 


5. N oisv data 

when "ideal" data is corrupted by noise or unwanted statistical 
variations, the collective parameters which characterize the process 
become functions of a random variable. The estimated averages will fluc- 
tuate according to how the data is taken and how much data is observed. 

The effect of noise can be reduced only by increasing the amount of 
data taken and applying averaging techniques or by smoothing the data 
record according to some a priori knowledge of the generating process. 
Quite often, data is smoothed by applying digital filtering to remove 
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the high frequency noise mixed with the desired data. In spectral analysis, 
noise effects are reduced both by selective filtering and by spectrum 
averaging. 

Selective filtering (also called notch, bandpass, highpass, lowpass, 
or rejection filtering) can be used when the desired spectrum is limited 
to some specified frequency range or ranges. This type of processing 
requires a knowledge of the "possible" spectral characteristics of the 
signal. Selective filtering and preprocessing of the data function were 
discussed i n Chapter IX . 

Spectrum averaging can be implemented when several spectral estimates 
for the same process are available. In the discussion of the Blackman- 
Tukey method of spectral analysis in Chapter IX, it was noted that the 
estimated spectrum is the convolution of the "true" spectrum with the 
spectral window function (Equation 9E.5). This is only true for the 
expected val ue of l^(f). In other words, l^(f), is a random function and 
several spectral estimates must be averaged before the composite estimate 
approaches the ideal case specified by Equation (9E.5). In this sense, 
even the smoothed estimate of the power spectrum given by E[t>(f)} is still 
a very idealized concept. 

There are essentially two ways to determine an average frequency 
spectrum in time series analysis. First, if the time series is used to 
generate a discrete autocovariance function which is then Fourier analyzed, 
the resulting discrete power spectral density function will be a set of 
spectral amplitudes and discrete frequencies. Several of these power 
spectral density records may be averaged to determine an average power 
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spectrum. In some instances it will be necessary to average the absolute 
values of the amplitudes to avoid the conceptual problem of dealing with 
theoretically impossible negative components. This can also be handled 
by windowing or other such techniques. 

Second, if the amplitude spectrum is calculated directly, the Fourier 
coefficients will be complex and one must chose one of two methods of 
averaging complex numbers. The first type of average is called the vector 
or coherent average. This processing involves averaging first the real 
parts and then the imaginary parts of all the complex numbers. It may be 
written as: 


IN IN 

Avefc } =- E a +j - E b 
n N k=l k N k=l k 


= a k + ^ 


(12D.26) 


It is called a coherent average because it is usually used when the phase 
information is important. The -second type of complex number average is 
called the algebraic or i ncoherent average. For this processing, one 
averages the absolute magnitudes and then the phases of all the complex 
numbers. The algebraic average is represented by 

+.oe 

Ave(c n } =c n e n (12D.27) 


where: 


c 

n 



N k= i 



e 

n 


1 N « 
- e e 

N k=l k 


(12D.28) 


(12D.29) 
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The equations for the vector or coherent average have been worked out 
using the familiar Fourier series notation given in Equation A2.2 in 
Appendix II. The reader is cautioned to realize that the time records 
must be phase connected or measured with respect to the same absolute 
phase reference or the advantage of a coherent average cannot be realized. 

A particular data set denoted by the subscript, m, is corrupted with 
noise and is written as 


x (k&t) =x(k&t) + e , 
m jek 


(12D.30) 


where the tilde accent denotes a measured value and e . denotes an additive 

mk 

noise time series for the m-th record. The estimated Fourier coefficients 
for the m-th record and the corresponding error contributions may be 
written as: 


k 

a 

mo 


+N 


2N + 1 


k=-N 


x(kAt) 

IQ 


1 +N 
2N +1 2 

k=-N 


x(k4 t ) 


2NtT 


+N 


Z 

k"-N 


'mk 


(12D.31) 

(12D.32) 


= x + As 


m 


a 2 ^ ^ 2 

a = 2N“+1 Z x (k^t) cos 
mn k=-N m 1 


a + Aa 
n mn 


4N 


mn = 


2^+1 2 x m (kAt) sin (? t L 

k-N m T 


“ b n + Ab mn 


(12D.33) 

(12D.34) 

(12D.35) 

(12D.36) 


(12D.37) 
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The coefficients a and £ are the estimates obtained without noise cor- 
n n 

ruption and are denoted by: 


* 2 21T 

a = 2INT+I E x(kAt) cos (— ^ nk^t) 
n k-N T 


( I2D .38) 


A 2 +N 2 

b = 2N— 1 £ x(kAt) si n (^r 1 nkAt) 

n f “ k-N 


(I2D.39) 


The contributions to these estimates caused by the noise corruption are 
given by: 


2 +N 2 

ia = 2N + 1 S « . COS (TIT nk^t) 
mn k=N mk 


(12D.40) 


1 +N 2 

^ b nm = Mil , * ^sinmi nk A t) 
k=-N 


I2D.4I) 


If there are MM independent records to be averaged, the average values 
for the estimated coefficients are calculated as: 

. ,A , _1 MM A 

Ave(a } -- 2 a 

mil mm r=l . m 


. — a + Aa 

n n 


(120.42) 


A 1 MM A 

Ave{b =- 2 6 

mil mm r = | m 


= l5 + Ab 

n n 


(I2D.43) 


The error term as r and A b R can be calculated from: 


MM 


■m 


1 L'JL'A 1 MM o TH a 

• k \ 4a m ' m 3ftT * C rk cos < 1 2D.44) 

r=l r=l k*-N 


-I MM 1 MM 2 -+N 211 

K = iS r E, 4l> m “ m r £, 2W+I k= 2 « rk cos <f nk4t ) (12D -45) 
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Actual data records are numerically processed by computing a mn and 
b mn for each record and then numerically averaging over all mm records to 
obtain Avefa } and Ave{6 }. The uncorrupted estimates of the spectral 
amplitudes are obtained by solving Equations I2D.42 and 120.43 to get: 


a n -Ave{^ n } - Aa n (12D.46) 

6 = Ave{b } - (12D.47) 

n mn n 


As the number of data points becomes very large, the estimates converge 

to the "true" values, a p and The error terms & a n and can be 

statistically specified if the statistical parameters of the noise series, 

e , , is known. This will allow one to obtain error limits and variance 
mk 

estimates on a and 6 • 
n n 

A similiar analysis for the algebraic or incoherent average can be 
derived. Starting with an absolute magnitude estimate for each spectral 
component of the m-th data record we obtain: 



(I2D.48) 


The incoherent amplitude average becanes: 


« 1 MM , 

A veC |c \} E lc I 

mn m -,i rn 


1 rn 


(1 2D .49) 


The reader should contrast this estimate with the coherent amplitude es- 
timate given by: 



(12D.50) 
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The advantage of a coherent average over the incoherent average for ob- 
taining an amplitude resolution limit will be discussed in the chapter on 
interpretation of estimated spectra. 


6. Detection of a single sinewave j_n bandlimited white G aussian noise 

A single frequency sinewave can be detected in bandlimited white Gaus- 
sian noise if the signal-to-noise ratio is large enough. In statistical 
detection theory (Whalen, 1971) the presence of a sinewave is determined 
by the change in the output of an envelope detector. Since the noise alone 
has a Rayleigh distribution and the signal plus noise has a Rician distri- 
bution it is possible to determine the probability of detection as a func- 
tion of signal-to-noise ratio for a given detection threshold. In spec- 
tral analysis the same types of effects are involved. A presentation of 
some basic ideas for statistical detection of periodic components in an 
estimated spectrum are presented in Chapter XIX. A simple presentation 
for a single sinewave and "ideal" noise will now be given. 

To begin, we define a time series that is the sum of a single fre- 
quency sinusoid and a white noise sequence: 


x(kAt) = 5(kAt) 


• * 2tt - . _ 2rr , . . 

+j — kAt -J — kAt 

r 1 * 1 t 

+ [ce + c^e ] 


(120.51) 


The white noise sequence (Appendix I, Part G) was defined as: 


E{5(kAt)} = 0, ECS(kAt)C<pAt)} = 0 , k £ p 

E{§ 2 (kAt)} = a| = P N 


(12D.S2) 


The questions to be answered are: 
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1. What are the important characteristics of the amplitude spectrum 

estimated from one or more sample records of the process? 

2, What is the effect of increasing the number of samples? 

Define a sample realization {x(kAt): k = o, +l,+2, ***,+n} of the time 
series in (12D.51) as (2N + 1) samples taken at an interval, At = I /1^ . As- 
sume that no aliasing occurs and that for any sample record the average 
value is approximately zero. A discrete Fourier transform applied to the 
sample time series will give (Equation 19D.3): 


A 

cn 


.. 2tt k 

i -t-fJ -J — — n 

2N + i £ ?(kAt)e -2N + 1 
k^N 


+ cjc— - nAf) +C X & (-^-nAf) 


2n 


{n=0,+l,+2, . . . , 4N } 


(12D.53) 


Representing the noise components with Equations 19C.3 

-j0. 

estimated amplitudes of c, in the complex form B en 


n - 


and 19C.4 and the 
gives: 
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C 

n 


1 

2N+ 1 




= D 


A 

O 

n 

2N+1 


B n co S « n ]-j[^ 


+ 1 


B sin 0 1 
n n" 


(12D.54) 

(12D.55) 


For a large number of samples the spectral mixing formula gives a 
good approximation for the amplitude C| • The large sample approxi mation 
i s 
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"j0i - J 

B le ^ Idle 

’j*_l +j e | 

B le =- I el I e 


(12D.56) 


and: 


B R =* 0 n t +1 


"n “2N + 1 ( *n ' j %) 


n ^ +L 

-je 


c l — 2N + 1 l c ll e 1 


(12D.57) 

(12D.58) 


If a large number of phase connected samples are available, a coherent 
average (Equation 12D.26) may be taken which yields: 


A 

Of 


A 

e 


Ave{cJ = - j E{t“y} = 0 


n 


2N + 1 

A 

a. 


(12D.59) 


Ave{c 1 } = E(2jJ7Y + l c i I cos 9 i}'j E ^2N+I + l c i I sin 9 i ^ 


= c 


(12D.60) 


This performance is very good but it takes large amounts of data. If the 
complex amplitudes are i ncoherentlv averaged the resulting estimate 


A 

or 


tL 


l = t<STT> 2 + <25Tl> 2 ^ »*±i 


(120.61) 


is Rayleigh distributed (Equation 19C.7). The resulting estimate for 
|cj has a Rician distribution (Whalen, 1971): 


^i i = [( 2i?Vr + l c il cos9 i )2+( 2NTr + l c ii sin0 i> 2]% ( l2D * 62 > 


2-i% 
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A statistical criterion for detection can be developed using the con- 
cepts of average spectral density and the Rician density function. The 
Marcum Q-F unction is used to test the value of IcJ against a threshold 
determined by Ave{£ n }. The author plans to develop this idea in a future 
paper. 

The same concepts can be applied to Equation I2D.54 the only effect 

_ J e -j<tl 

is the change of Idle 1 to Ble caused by the spectral mixing effect. 

The reader is referred to Chapter XIX, Section C where more properties 
of the DFT and Gaussian noise are discussed. In particular, a relatively 
simple criterion is given for testing a noise spectrum to see if it con- 
tains a periodic component. 
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XIII. PREDICTIVE DECONVOLUTION 

An analog or digital filter convolves an input, x(t), with the filter's 
impulse response, h(t), to give an output, y(t). The central problem of 
deconvolution is: given the output y(t) and either x(t) or h(t), can the 

third function be uniquely determined? The answer is very definitely no. 
Deconvolution involves finding the inverse filter for h(t) and this demands 
that the reciprocal of the transfer function of h(t) exist for all frequen- 
cies of interest. 

To provide a practical basis for the problem of deconvolution it is 
necessary that the class of possible filters be restricted to those for 
which the inverse filter exists and is well-behaved. This restriction may 
appear to be too severe but it turns out that much scientific data process- 
ing and spectral analysis problems can be modeled with such a class of 
filter. A class of filters variously called prediction error f i I ters. 
digital W iener f i I ters. least-squares i nverse f i I ters. least-squares decon- 
vol uti on f i I ters, prewhitening f i I ters, or recursive digital fi Iters have 
been successfully used in the deconvolution of geophysical data (Peacock 
and T reitel, 1969). 

Predictive deconvolution. techniques are usually applied to a degen- 
erate class of time series characterized by a known component and a purely 
random component. The desired objective is to take a measured time series, 
x(t), and decompose it into a completely predictive component, z(t), and a 
random or uncorrelated component, r|(t). 
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In the analysis of practical time series and especially when applying 
autoregressive analysis techniques, it is assumed that the time series 
(z(t): t =0, ±1, ±2, •••} can be characterized by a finite number of unique 
parameters (these will be discussed shortly). The uncorrelated component 
for these analysis techniques is represented by the white noise sequence 
t =0, 1, 2, •••}. Characteristics of a white noise process with 
finite variance are discussed in Chapter XIV. 

For the rest of this chapter, the time index (t=0, +1, +2, •••} will 
be used as shorthand notation for {t=kAt: k =0, +1, +2, •••}. This nota- 
tion is common in the literature and so is the convention of assuming 
At=l. This convention will be used except where the explicit omission of 
At may cause confusion. 

The three most commonly used mathematical models for representing a 
measured time series are the Fourier series, the autoregressive series, 
and the convolution of the known and random components. The Fourier series 
representation may be written as the sum of a white noise sequence (see 
Equation 14.3) and a complex Fourier series with complex amplitudes, c n 
and Fourier period Tp: 

+j 2 IJ n t 
■HI T f 

X(t) =?((:)+ See =f(t)+z(t) (13.1) 

n^M n 

The predictable component, z(t), is characterized by the 2M + 1 complex 
amplitudes and the Fourier period, Tp • This constitutes the finite number 
of unique parameters for this representation. 
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The second representation is an autoregressive series (see Chapter XIV 
and Appendix I) of order K: 

K 

x(t) =f(t)+ E a x(t - n) (13.2) 

n=l n 

The predictable component is characterized by the finite number of ampli- 
tude coefficients [a : n=l,2, •••K}. The power spectral density function 
for this process model is both continuous and bandlimited. The resulting 
autocovariance function is infinite and nonperiodic. 

It is reasonable for the reader to wonder at this point if a process 
can be modeled as the sum of a Fourier series and an autoregressive series. 
The answer is, yes, a process that has a power spectral density function 
that has both continuous and discrete components should be represented as 
the sum of both models (Cooper and McGillem, 1971, Ch. 6). The autocor- 
relation function for such a model will be infinite and contain periodic 
and nonperiodic components. 

The third common model is the convolution of a white noise sequence, 
r|(t), with a completely known time series, z(t), to give: 

y(t) = z(t) * r|(t) = E z(n) t) (t - n) (13.3) 

n 

It is this model that is used by Peacock and Treitel (1969) to develop a 
scheme of predictive deconvolution. The notation y(t) is used for this 
model to avoid confusion with the autoregressive process model which yields 
similiar results. 

The desired least-squares deconvolution filter which characterizes 
z(t) is the one which ideally transforms y(t) into a white noise sequence 
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and hence 


y(t) b(t) = z(t) *rj(t) * b(t) =r|(t) 


(13.4) 


or: 


y(t) *b(t) =n(t) (13.5) 

The deconvolution filter and the predictable component are now related by: 
z(t)*b(t)=l (13.6) 


If both z(t) and b(t) are modeled using predictive filters, b(t) can be 
referred to as the inverse filter characterizing z(t). This is why the 
technique is often called least-squares inverse filtering. Peacock and 
Treitel (1969) develop a general technique whereby the data function yet) 
is used to generate a filter b(t) which describes the deconvolved component, 
z(t). z ( t) can be described as a filter or convolving function which 
transforms a white noise input, rj( t), into the time series, yet). What 
this really means is that the spectral function representing yet) is 
I i mi ted to a class of functions that can be descri bed as the output of a 
real f i I ter with white noise input. Also, the inverse filter must exist. 
Before continuing, we will digress to examine Equation 13.1 and to put it 
into an alternate but especially interesting form. First substitute 
Equation 11.12 into (13.1) to get: 


. 4M +N 

x(t) = §(t) + 2N ~ +T 2 2 x < k At)e 

n=-M k=-N 


. 2rr 2rm 

-j — nkAt +j- — t 

F F 

e 


(13.7) 


By interchanging summation this can be written as: 
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+N 


+M 


J n(kAt _ t) 


x(t)=S(t) + S x(kAt){2N+l E e 
k=-N n— M 


} 


(13.8) 


Now the kAt dependence is factored out of the term in brackets and the 
series is written in a more compact form as 


+N 


x(t) = |(t) + Z a (t) x (kAt) t = 0, +At, +2 At. +NAt. (13.9) 
k-N * 


where: 


^t) - 


1 


2N+1 


^ -j n(kAt - t) 

E e F 
n=-M 


(13.10) 


1 M 2 

= 2N + 1 (1 + 2 Z coseTu n(kAt - t) 1 > 
n=l F 


(13.11) 


It is interesting to note that (13.9) is similar in form to (13.2) except 
that the coefficients, ak(t), are functionally dependent on the index t. 
Also, the coefficients are even in both t and kAt and are another form of 
the estimated Kronecker delta function (Equation 11.19). This difference 
between (13.2) and (13.9) accounts for the difference in the two types of 
time series representations. The indices for the representation of (13.9) 
do not have to be symmetrical about t =0 and could just as well be written 
as 

2N 

x(t) =|(t)+ E c&(t)x(kAt) t = 0, At, ••• 2NAt (13.12) 

k=0 K 

with c^Ct) now being symmetrical about the point, t=(N+l)At. 

If the time series contains a single frequency spectral component 
where the period is fairly well-known, the complex Fourier amplitude can 
be obtained by using the estimator (Equation 11.15): 



195 


i 1 +N To 

C(fJ =2N + 1 S x(kAt)e (13.13) 

0 k=-N 

This component is then removed from the data by (See Appendix II) 

x(t) =x(t) - 2|<*(f 0 ) Icos (T 2 " t + 0 O ) (13.14) 

o 

where <t> Q is the phase angle of <$(f 0 ), The numerical algorithm for this 


estimation will take the form: 

. 1 2N + 1 2 

Re{C(f )}=2N +1 £ x(kat) cos [In (k- N- l)&t] (13.15) 

k=l o 

, 2N+1 9tt 

I m Cc(£ 0 )} = 2N + 1 2 x(kAt) sin t 11 (k-N - l)At] (13.16) 

k=l o 

|<$(f Q ) | - [Re 2 {c} + Im 2 tc}] % (13.17) 

0 = arctanE^-4 (13.18) 

° Re{<*} 

x(kAt) =x(kAt) - 2 |^(f Q ) Icos (k- N- l)At + 0 o ] (13.19) 

o 


PROGRAM 06 in Appendix III has been written to implement the estimator 
described above. The modified time series, x(t), may also be spectrum , 
analyzed to determine a "residue" spectrum, if the period T q is not 
exactly known, the complex amplitude can still be found by using approx- 
imation techniques. For example a discrete Fourier transform estimation 
using x(t) would give approximate spectral components to use as starting 
values. 
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We now proceed to derive the mathematical properties and methods 
involved in determining a suitable predictive deconvolution technique. 

The predictive filtering viewpoint and the autoregression analysis view- 
point will both be presented since they are mathematically equivalent. 

The discrete convolution formula for a digital filter (Equation 5C.4) 
i s 

t 

yet) = ih(t-n)x(n) t = 0, +l,+2, ... (13.20) 

n 

where the lower range of the index, n, depends on the length, K - 1, of the 
digital filter. The sampled impulse response of the filter is limited in 
time to the series {h(nAt): n=0, 1, 2, ••• (K -I)} where h(nAt) =0 for 
k^K. K is called the length of the filter and the lower limit on n 
becomes (t- K + l). The filter coefficients have been normalized by At. 
Linear prediction can be formulated in terms of a filter output by con- 
sidering h(t) to be a linear prediction operator with prediction distance 
a At. The output of the filter will be an estimate of the input time series 
at some future time t + <*. This is represented by: 

t A 

yet) = £ h(t- n) x (n) =x(t+a) (13.21) 

The best linear prediction in a least-squares sense is made by a filter 
with coefficients that minimize the total mean-square prediction error: 

M = £ e 2 (t + a) =£[x(t +cx) - x A (t +a)} 2 t =0, ±1 , ±2, ••• (13.22) 

t t 

The error of the prediction, s(t + o’), is the difference between the "true" 
and predicted values of the time series: 

A t 

e(t-t-o') =x(t+a; - x(t-l-a) =x(t+a) - £ x(n) h (t - n) 

n 


(13.23) 
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The total mean-square error can be written in terms of the infinite time 
series and the filter coefficients: 

t 2 

M= £ [x(t+a) £ x(n) h (t- n)] (13.24) 

t n 

A system of K equations can be used to determine the K unknown filter co- 
efficients that will minimize M. This system of equations is obtained by 
the following minimizations: 


|(k) =0 k=0,l,2, (K- 1) 


The k-th linear equation is derived from 


ii(k) = ' 2 s Cx(t + or) - Ex(n) h (t - n)J x(t - k) =0 
This may be written in the more usual form as 


£ [ £ x(n) h (t - n)]x (t- t) = £ x( t + a) x (t-T) 
t n t 

"=0,1,2, •••, K-l. 


(13.25) 


(13.26) 


(13.27) 


or: 

K-l 

£ h(n) r(T-n) = £ x(t + a) x (t - t) (13.28) 

n=0 t 

t — 0 j 1, 2, •••, K- 1 


If we make the following substitutions 


r(T-n) = £ x(t) x (t - t+ n) (13.29) 

t 


r(x + a) = £ x(t + or) x (t - t) 
t 
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we can. write the system of equations given in (13.28) in terns of the co- 
variance matrix and filter coefficients as (remembering r(r)=r(-T)): 

reO) r ( I ) r(K - 1) h(0) r(a) 

r(i) reO) ••r(K-2) h(l) r(a+i) 

r ( 1 ) 

r(K-l) r(K -2) •• r(l) reO) h(K-l) r(crfK-l) (13.30) 

The covariance matrix is made up of elements: 


r(T) = £ x(t) x (t - t) = r(-T) (13.31) 

t 

The system of equations represented by (13.30) is solved for the filter 
coefficients by inverting the covariance matrix. The least-squares pre- 
diction operator represented by h(t) can be used for the prediction: 


A 1 

x(t + a,) = £ h(t - n)x (n) 
n 


(13.32) 


The prediction error operator is defined as 

a - 1 zeros 

( g ( n ) } = ifo^TTo: _h(0), -h(l ), •••, -h(K-l). 
= geO), g(l), •••, g(a+K). 


which may be used directly to obtain the error of an a-step prediction: 

cr+K 

e(t + a) = £ g(n) x (t + a- n) (13.34) 

n=0 

Equation 13.34 is also an alternate representation of (13.23). 

The least-squares prediction operator characterized in Equations 13.30 
and 13.32 contains the elements of a prediction f i I ter while the least- 
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squares prediction error operator of (13.33) represents a prediction error 
f i Iter. Before looking at the central problem of predictive deconvolution 
and filtering we will digress to consider the autoregression approach to 
the same analysis. 

For autoregressive processes modeled by (13.2), the best linear one- 
step predictor in a least-squares sense (Appendix I and Koopmans, 1974) 
is given by: 

A < 

x(t+l ) =x(t+l )-g(t+l ) = Z ax(t+l-n) (13.35a) 

n=l n 

= a 1 x(t) +a 2 x(t-l) + x(t - K) (13.35b) 

The one-step prediction error is the expected value of the mean-square 
error: 

E [«x(t+l ) -x(t+l » 2 > =E (t-hl ) >=c 2 (13.36) 

Koopmans (1974) has shown that the best a-step predictor in a least-square 
sense for an autoregressive series is (o-^K+l): 

a °'" 1 a K 

x(t+a)= Z ax(t + or-n)+ Z ax(t+a-n) (13.37) 

n=l n n=a n 

x(t+a)=a| x(t+a-l )+a 2 x(t+a-2)+ ••• aa l x(t+l) 

+ a Q x(t) + a Q+1 x(tr 1) + ... a- K) (13.38) 

The a-step prediction error is (0’s2): 

a 2 2 2 a -1 2 

E£(x(t + or) -x(t+a» } = E[| (t + or) } = a [ 1+ Z a_ ] 

n=l n 


(13.39) 
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The coefficients [a Q : n=l, 2, 3, ••• K} of the autoregression can be ob- 
tained by solving the matrix of autocovariances specified by the Y ule- 
W alker equations and derived i n C hapter XIV (Equation 14.47). In the 
notation of this chapter, the solution is: 


reO) 

r(l) 

• • r(K -1 ) 

-1 

r ( 1 ) 

r( 1 ) 

reO) 

•• r(K -2) 


r(2) 

r(K-l) 

r(K -2) 

reO) 


r(K) 


(13.40) 


For a one-step predictor (<*= 1) Equations 13.40 and 13.30 give the 
same coefficients. The relationship between the filter coefficients and 
autoregressive coefficients is expressed as: 


hen -1) =a n=l,2,3, •••,K (13.41) 

Although we see that the predictive filter and autoregression are two 
seemingly separate viewpoints on prediction and prediction error for time 
series, they produce identical results for the one-step predictor. This 
equivalence will become important later on when maximum entropy spectral 
analysis is derived in terms of predictive filters and autoregression. 

The deconvolution operator, bet) is determined using the one-step 
prediction operator and the condition given by Equation 13.6. The basis 
for the existence of this operator is the fact that the autocorrelation 
function for yet) differs from that for s(t) only by a constant scale 
factor. This will now be shown. Using an analogy to Equation 5B.8, we 
know that the spectral density of yet) is equal to the product of the 
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spectral densities of z(t) and r)(t) 


S (w) = S (w)S (w) (13.42) 

y z h 

and consequently the convolution of their autocorrelation functions is: 

V T| - R zz (T) *% (T> < 13 ' 431 

From theory we know that the autocorrelation function for white noise is a 

2 

delta function with strength g : 

R (T) = a 6(t) (13.44) 

The convolution integral of (13.40) becomes 

V T) =G l R zz (T) (13 ' 45) 

and we see that the output and input autocorrelation functions differ only 

2 

by a constant amplitude scaling factor, a . We can also conclude that the 
power spectral density function amplitudes differ by the same scale factor. 

The property of (13.45) has been used by Peacock and Treitel (1969) 
to develop a predictive filter for deconvolving z(t) and rj( t) . The matrix 
of (13.30) is augmented by adding the estimated variance of the white noise 
sequence (see Equation 14.44) 


2 K 

cr =r(0) + £ r(n) h (n - 1) 
n n=l 


K 

= R (0) + £ R (n) h (n - 1) 
YY n=l YY 


(13.46a) 

(13.46b) 


to give: 
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fr<0) 

r( 1 ) 

r(K ) 


l 

2 

a 

T1 

1 r(l ) 

r(O) 

r(K-l) 


'h(O) 

0 





-h(l) = 

0 

r(K) 

r(K-l) 

r(O) .. 


-h(K-l) 

0 

yy 



(13.47) 


The column vector [h] forms a one-step prediction error operator which 
operates on [r] yy to give the one-step prediction error, The aug- 
mented matrix equation (13.47) is identical in form to that determined by 
augmenting (14.46) or (13.40) to give: 


|r(0) r( 1 ) f(|(' 

| — 


5 


2 

r( 1 ) r(O) r(K-l) 


- a l 


0 

• 

• 



" a 2 

— 

0 

• 

r(K) r(K-l) r(O) _ 

/y 



0 


Again we see that predictive filtering and autoregression are equivalent 
viewpoints. By comparing (13.48) and (13.47) with (13.5) and (13.6) we 
see that the deconvolution filter or predictive deconvolution operator is 
characterized by the following system of equations: 


r(O) 

r(l) 

r(K ) 


b 

o 


1 

r(l) 

r(O) 

. . r(K-l) 


b l 


0 






= 

0 

r(K ) 

r(K-l) 

r(O) , 

'y 

b k 


0 


(13.49) 
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The deconvolution operator is seen to be just a scaled version of h(n) or 

a as: 
n 

b o = "T 

a S 

b n =- a n b n = hen- 1) 

Deconvolution is performed using Equation 13.5. The power spectral density 
function for y(t) is characterized by z(t) or the inverse of bet). This 
will be shown in Chapter XIV. 


b o=-T 

cr 

n 


( 13 . 50 ) 
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XIV. AUTOREGRESSIVE SPECTRAL ESTIMATION 

The estimation of a continuous power spectral density function using 
autoregressive techniques is based on the fact that any weakly stationary 
(covariance stationary) stochastic process with a continuous spectrum can 
be represented by an infinite moving average (Koopmans, 1974). 

A moving average process is defined by the series 

+=> 

yet) = E bC(t-n) t=0,±l,±2, • (14.1) 

n=.® n 

where the coefficients, b n , are restricted to: 

+• 2 

E b < ® (14.2) 

_ n 
n=-“ 

The time series [y (t): t =0, ±1, ±2, ••• } is completely characterized by 
the coefficients (b n } and can be thought of as a linear transformation of 
the white noise process represented by §(t). The white noise time series 
{§(t) : t =0, ±1, +2_,. ••• } characterizes the white noise process and is 
completely described by its mean, autocovariances, and spectral density. 
The mean of the white noise process is zero 

E[?(t)}=0 (14.3) 

and the autocovariances are: 

a 2 k = 0 

C(k) =E{§(0 ? (t-k)} = 1 (14.4) 

{ 0 , k^O 


The power spectral density function is the Fourier transform of the auto- 
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covariance function: 

+03 

S (a) = At E C(k)e" ju,kAt = CT 2 At (14.5) 

~ k=-® 

The reader should be made aware that the notation in this chapter 
will differ from that in Appendix I. Also, subscripted variables and 
functions may appear with actual subscripts or with an index in parentheses. 
The sampling interval in. all equations will be represented by At. 

For practical analysis considerations, the time series generated by 
a moving average process is one-sided: 

K 

y(t) = s b S(t-n) (14.6) 

n=0 n 

The process and the resulting time series are characterized by the K + l 
coefficients (b : n=0, 1, 2,3, ••• K }. The value of y(t) depends only 
upon the b n s and K + l present and past values of §(t) • 

The time-series for a one-sided moving average process can be synthe- 
sized numerically by first generating the white noise sequence with a ran- 
dom number generator and then doing the linear operations described by 
Equation 14.6. This gives 

y(t) = b o §(t) + b 1 §(t-l) + ... + b K §(t-K) (14.7) 

for al I t =0, ±1 , ±2, •••• 

The power spectrum for a moving average process is obtained by using 
a digital filtering analogy. A digital filter convolves the white noise 
sequence with the filter coefficients {b n } to produce the output time 
series. Since the spectrum of the input white noise sequence is flat, the 
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spectrum of the output time series will be the amplitude response of the 
filter multiplied by the constant input spectrum. 

A look at the convolution and z-transform properties of a digital 
filter will help provide better understanding of the autoregressive spec- 
tral estimator. The output, y(t), of a digital filter is the convolution 
of an input time series, x(t), with the filter weights [b^}: 

+=> 

yet) = Z bkx(t- k) t = 0, +1, +2, . . . (14.8) 

k =->co 

To obtain the filter weighti na sequence we define the impulse response of 
a linear filter, get), as the filter output in response to a delta function 
input. In the sampling domain, the sampled impulse response or weighting 
sequence is: 

b = 7 - g(nAt) (14.9) 

n At 

From z-transform theory we know that the convolution theorem in the trans- 
form domain (Equation SA.8) is: 

Y (Z) =G(Z)X(Z) (14.10) 

The next step is to define what is meant by the transfer function, G(Z). 

From basic filter theory we know that a complex transfer function, 

G( uj) , is defined for sinusoidal inputs (x(t) = sin tut) by using the relation 

yet) = IG (w) I sin (tut + 0(to) 

and for complex sinusoidal inputs (x(t) =e +jat ) as: 


(14.11) 



y(t) = |G(uu) j e + j 0(uj) e +:iU£ (14.12) 

In the same manner, the transfer function for the digital filter is ob- 
tained by applying a complex sinusoidal input (x(t) = e+j wt) and factoring 
the output into the product of the input and a transfer function as shown 
in (14.12). When this is done for the convolution sequence given in (14.8) 
the result is: 

yet, = r r v' 3 “ kAt < i4 - i3 > 

k=-“ k=-“ k 

The complex transfer function for the digital filter can now be identified 
as: 

G(W) = b e-jwkAt (14 14) 

k=_CD k 

The filter weights or sampled impulse response can be determined from the 
transfer function by the transform: 


b . 1 J 4)T G(w)e + j IoWt dw (14.15) 

k 2TT * 

-TT 

The single-sided z-transform for a causal filter is determined from (14.14) 
by substituting Z=e + 3 u ^ t to generate the Z-t;ransform of the filter 
weighting series: 

K 

G(Z) =B(Z) = E b Z-n (14.16) 

n=0 n 

The continuous spectral density for the moving average process can 
now be determined by driving the digital filter with white noise and 
determining the output spectrum. The relationship between input and output 
spectral densities (Equation 5B.8) becomes: 
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S (w) — I G (w) I2S (w) 

y A 


= |g(cd) | 2 S_(tu) 


(14.17) 


Using (14.5). and (14.14) the final result is: 


+® 


S„« = Ato‘| I b k e-J“ kit 


k=-CD 


(14.18) 


Likewise, the power spectral density for the one-sided process becomes: 


s 

y 


(id) = Ata 


K 

£ 

n=0 


b e -ju)kAt 
n 


2 


(14.19) 


The continuous spectral density can also be obtained by evaluation of the 
z-transform on the unit circle: 


S y (w) =Ata 2 |B(Z) l 2 , Z = e +ju * t , Iwl (14.20) 

The continuous spectral density function for a moving average process is 
periodic with period 2m At. Usually only the principal spectrum, Iwi sHlAt, 
is of interest. 

We now define a time series [x(t): t=0,±l,±2, ••• } as representing 
a weakly stationary (covariance stationary) finite autoregressive process 
if it satisfies the K-th order autoregressive series: 

K 

E ax(t-n)=?(t) t = 0, +1, +2 . . . (14.21) 

n=0 n 

The autoregression must satisfy the conditions (Koopmans, 1974) 


e{x(s) ? (t) } = 0 foralls<t 


(14.22) 


and: 
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a =1, a^O, s a^<» (14.23) 

0 K n=0 n 

The finite autoregressive series is now written as: 

K 

x(t) - s a x(t- n) = |(t) (14.24a) 

n=l n 

x(t) - alx(t - 1) - a 2 x(t - 2) - ••• - - K) = §(t) (14.24b) 

The power spectrum for a finite autoregressive series can be obtain 
both from a digital filtering analogy and from a one-to-one correspondence 
with a moving average process. A digital filter operating on the time 
series (x (t) : t =0" ±1, ±2 ••• } produces the white noise sequence 
{|(t): t =0, ±1 , ±2" ••• }. This special type of filter is referred to as a 
prewhitening filter by most authors (see Chapter IX). It is a simple 
matter to define the relationship between input and output spectral densi- 
ties as 

S x (o) |A(uu) 1 2 = s ^ ( oj) (14.25) 

and consequently: 

2 

S (w) = 1 ? S (w) = (14.26) 

IA(w) 1 * |A(<i>)| 

The z-transform of the digital filter is 

K 

A (Z) = S a Z' n (14.27) 

n=0 n 

and the transfer function in the frequence domain is obtained by evaluating 
A (Z) on the unit circle: 
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IA(W) I , IA(Z) II - I l .e-i™ 1 ' 1 (14.28) 

z l e +mt n=0 n 

Koopmans (1974, p. 218) has shown that if A (Z) has no zeros on or outside 
the unit circle, the linear filter can always be inverted and llA(w) is 
square integrable. 

The equivalence of a finite moving average and a finite autoregressive 
series can be shown by applying a Fourier series expansion for the inverted 
filter and constraining the Fourier coefficients to those which will sat- 
isfy the boundary conditions of Equations 14.22 and 14.23. The infinite 
moving average formulation is 

= r b k e-3“ k = |g(id) |e + *<“> ( 14.29) 

and it follows from Equations 14.1, 14.12, 14.13, and 14.14 that the finite 
autoregressive series can be expressed as a moving average: 

+® 

x ( t) = Z b S(t- k) (14.30) 

k=-® 

In order for the moving average to satisfy the boundary conditions in 
Equations 14.22 and 14.23 it is necessary and sufficient (Koopmans, 1974) 
that: . 


E (x(s) § (t) } =0 (14.31) 

for s = (t- 1), (t- 2), ••• This can be shown by taking the expected 


value using Equation 14.30: 
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+® 

E(x(s)?(t)}= S b k E{?(s- k) ? (t) } (14.32) 

k=-® 

Using the identity, e{£(s - k) % ( t) } = 6(s - j - t)a 2 , (14.32) can be further 
reduced to show: 

2 2 

S b k 5(s-j-c )<j =b t o =0 (14.33) 

k=-eo 

and this implies the boundary condition: 

b-1 =b-2 =b-3 =••• =0 (14.34) 

For s =t, the expected value in (14.32) becomes 

E ( x ( t) | (t)}=b o a 2 (14.35) 

but from Equation 14.24a we know that 

E{x(t) § (t)} = E{§ 2 (t)}= a 2 (14.36) 

so that b Q =l is the final boundary condition. 

In conclusion, a weakly stationary time series satisfying Equations 
14.24 will have a one-sided moving average representation given by: 

K 

x (t) = 1 + E b |(t “ n) (14.37) 

n=l n 

For a covariance stationary finite autoregression to exist, it is suffi- 
cient that all the zeros of B(Z) lie inside the unit circle. The z-trans- 
form of a one-sided moving average is related to the z-transform of the 
autoregression as: 


A (Z) = BIZI 


(14.38) 
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The continuous power spectral density function for a finite 
regression is written as: 


or: 


s (U)) = CT At|l + Z 

x n=l 


b e -juun At .2 
n 1 


S (w) 
X 


2 

q At _ _ _ _ 

11 - 2 *^ e "J wn P 

n=l n 


The stationary autocovariances for the autoregressive process are 
as: 

C(k) =E (x(t) x (t- k) } = C(-k) 


The Y ule-Walker equations (Equation A 1.31) relate the autoregressi 
efficients, a n , and the stationary autocovariances: 

K 

C(k) = Z a C(k - n) , k =1, 2, —K 
n=l n 


C(k) =alC(k- 1) +a 2 C(k- 2) + a^Ck- k) 

A useful estimate of the variance of the white noise sequence can 
tained by combining Equations 14.36 and 14.37 to give: 

E (x(t) | (t) } =E [x( t) Z a x(t - n)} = Z a C(k) =ct 

n=0 n=0 


This estimate may be written as: 

2 K 

a =C(0) + Z a C(n) 
n=l n 


auto- 

(14.39) 

(14.40) 

defined 

(14.41) 
on co- 

(14.42a) 

(14.42b) 
be ob- 

(14.43) 


(14.44) 
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To obtain the estimated power spectral density of a finite sample 
{x(t) : t=0, 1,2, ... (M-l) } of the "true" time series (x(t)}, the estimated 
autocovariances, <5(k), and the estimated autoregressive coefficients, an' 
must be determined by appropriate numerical estimation procedures. 

One commonly used approach to this estimation problem is to compute 
estimatedautocovariances from the definition: 


a M-k a a 

C(k) _ m 1 k E x(t)x(t- k) 
— - t=0 ■ 


(14.45) 


Next the Y ule-W alker equations are used to define a linear set of equations 

in <£(k) and a : 
v n 


< 5 ( 1 ) 


<5(0) <5(1) 

<5(K?1) 

<5(2) 

= 

<*(1) <5(0) 

<5(K-2) 

< 5(*0 


<5(K-1) <?(K-1) ••• 

ho) 


3 1 

a\ 


A 


(14.46) 


For initial calculations, the order of the process has been estimated as K. 
The KXK covariance matrix is inverted and the autoregressive coefficients 
are computed from: 


A 

3 1 


<5(0) 

C(K-l)' 


0(1) 


A 

a 2 

= 

<5(i) 

<5(K-2) 


(5(2) 




1 

H-* 

v-/ 

• 

1 

• 

<5(0) 



(14.47) 


The spectrum of the estimated white noise sequence is generally not a con- 
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stant unless the sample is very large. The estimated autocovariance 
function for the approximate white noise sequence is written as: 

E^(SO =Ef|(t) £ (t- k) } (14.48) 

Substituting the autoregressive series of Equation 14.24a for both £(t) 
and |(t - k) gives 

R (k) = E{S a x(t-n) s ax(t-p-k)} 
n=0 n p=0 p 


= £ a £ a E {x(t - n) x (t - p - k) } 
n=0 n p=0 p 

K K 

= E S a a <$(n- p+k) (14.49) 

n=0 p=0 n p 

for k =0, ±1, ±2_, ••• ± (M - K) • 

The estimated spectral function ^ (u>) can be obtained from ^ (k) by using 
a lag window function and a digital Fourier transform. A spectral density 
estimate for the white noise sequence can also be obtained using Equation 
14.44 to give: 


a 2 =& a A £(n) (14.50) 

n=0 n 

The estimated spectral density for these numerical techniques can be 
written as 


£ (w) 
X 




2 

11- s a e-] wn 1 
n=l ° 


(14.51) 


or: 
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g At 

|1-S ie- JamAt 

n=l n 


( 14 . 52 ) 


Other methods of obtaining best estimates of <$(k) and a n may also be used 
effectively. Most methods employ a least-squares estimate that also con- 
strains the covariance matrix so it will remain positive definite. This 
will constrain the spectral estimators to be positive over the entire 
frequency range. More about these estimators and the proper constraints 
will be given in Chapter XV. 
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XV. MAXIMUM ENTROPY SPECTRAL ESTIMATION 
A. Introduction 

A new criterion for estimating the continuous power spectral density 
function for a random time series (Stanley, 1975) was first proposed by 
Burg (1967). This estimator, called the Maximum Entropy Method (MEM), 
essentially maximizes the average entropy (Newman, 1977) over the informa- 
tion bandwidth using the estimated autocorrelation function. MEM was de- 
veloped as a technique to minimize the effects of observer bias and reduce 
certain types' of processing error on the spectral estimates. For example, 
one of the most noticeable differences between MEM processing and other 
commonly used techniques is that no window function selection is required. 

It is important for the reader to realize that MEM spectral analysis 
is not meant to be a replacement for other techniques and it is not vastly 
superior to others. As with any estimation technique, MEM has its own 
strengths and weaknesses based upon an assumed process model. When reading 
the published literature on MEM it is most important to carefully assess 
the various assumptions and boundary conditions used to derive the pub- 
lished results. 

As discussed previously in Chapter IX, all spectral estimators must 
use a particular model to fit empirical data. In addition, most apply 
smoothing and all apply extrapolation or a periodic extension to the data 
to obtain infinite data functions which conform to the selected criteria. 
Once an infinite estimate is obtained, the spectral density can be found 
using basic theory (Chapter IV). A summary of the commonly used models 
will help to put mem spectral analysis in proper perspective. 
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The model for Fourier transform spectral analysis is based upon a 
periodic extension of the empirical data function. Constraints on this 
extension include assuming that the observation period TN is equal to the 
Fourier period T , that the signal is bandlimited, and that the number of 

r 

discrete spectral components does not exceed the number of data samples 
(adequate sampling). This model gives excellent results for many applica- 
tions but can have serious deficiencies if the true process is not periodic. 
In Fourier transform spectral analysis only the amplitude spectrum is 
usually computed because the power spectrum is a trivial extension (Chapter 
XII, Section C). Many users object to using a periodic extension assump- 
tion and an amplitude spectrum for analyzing data. These users often 
prefer the B lackman-Tukey method. 

The B lackman-Tukey model for spectral analysis assumes a zero exten- 
sion of the estimated autocovariance function. This zero extension assump- 
tion is emphasized by the use of a lag window function which guarantees a 
zero value beyond some chosen lag value t . Although this assumption is 
often unrealistic, it gives good results in many applications. Users of 
the B lackman-Tukey method most often complain about lack of spectral 
resolution, inefficient computation and use of empirical data, and nega- 
tive spectral components. 

A rigorous discussion of the properties of the various spectral esti- 
mators was given in Chapters VIII, IX, and X. As a rule the most commonly 
used power spectrum estimators compute an estimate of the autocovariance 
function, apply a window function to this estimate, and obtain the power 
spectrum from the Fourier transform of the result. This technique produces 
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an estimate that has the following deficiencies: 

1. The inverse transform cannot reproduce the original estimated 
autocovariance function. 

2. The lag window introduces spectral smoothing which decreases 
spectral resolution. 

3. The lag window may produce a modified autocovariance function 
which could result in negative spectral components. 

4. Most estimates of the autocovariance function use less data as 

the lag value increases thereby increasing statistical variability. 

5. The estimate of the autocovariance function may be badly truncated. 

6. The arbitrary choice of a lag window function may tempt the ob- 
server to choose the one that produces the "most agreeable" 
spectrum. 

All of these deficiencies can be minimized or eliminated by using the MEM 
spectral estimator. The following results are obtained with MEM: 

1. The estimate is constrained so that the inverse transform will 
exactly reproduce the original estimated autocovariance function. 

2. MEM does not explicity use a lag window function and empirical 
test results show it produces a frequency resolution superior to 
other methods when short data records are employed. 

3. MEM constrains the autocovariance matrix to be nonnegative def- 
inite so that only positive spectral components are produced. 

4. MEM incorporates nearly all of the input data function for each 
step in the autocovariance estimate and hence reduces the problem 
of statistical variability for large lag values. 
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5. MEM produces an estimated autocovariance function that can be ex- 
trapolated to any desired order of lag and consequently eliminates 
the truncation problem. 

6. Since MEM is a data adaptive method, the selection of a lag win- 
dow is implicit in the process and its use is transparent to the 
observer. 

The maximum entropy criterion for spectral analysis will not reduce the 
intrinsic spectral estimation errors (Chapter IX) of aliasing, statistical 
variability, and lack of stati onarity. These continue to be important 
practical problems associated with all spectral estimation methods and 
must be dealt with on an individual basis. 

The Maximum Entropy Method of spectral analysis has the following 
limitations and known deficiencies: 

1. The validity of the spectral estimate relies on the assumption 
that the measured process can be adequately modeled by a finite 
autoregressive series. This requires a "noise-like" data function 
and a spectrum that is both continuous and bandlimited. 

2. The observer must select an arbitrary order for the autoregressive 
process that is used to model the data. This selection intro- 
duces observer bias into the estimate because it controls the 
amount of spectral smoothing and hence resolution. A nonsub- 
jective solution to the problem of selecting an optimum order has 
not yet been solved (Parzen, 1969). 

3. For estimations using a large order autoregression, the MEM algor- 
ithm may become numerically unstable. This occurs because the 
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locations of the zeros of the characteristic polynomial shift 
slightly due to numerical truncation and fall very near or on the 
unit circle. 

In spite of these deficiencies, the MEM technique represents a powerful 
tool for the estimation of time series spectra. Along with FFT, Blackman- 
Tukey, and autoregressive techniques it should become a widely used method 
of data analysis. Its properties and applications will now be presented 
along with some derivations of fundamental concepts. 

B. Fundamental Descriptions for MEM 

The Maximum Entropy Method can be simply described as a technique for 
obtaining an infinite autocovariance function from finite data. The em- 
pirical time series is modeled with a finite order autoregressive series 
and the resulting spectral estimate is constrained by maximizing the 
average entropy. Practical algorithms may compute either the estimated 
autocovariances or the autoregressive coefficients. The power spectral 
density function is obtained by taking the Fourier transform of the auto- 
covariance function or by using the autoregressive spectral estimator. 

More than one viewpoint can be used to summarize the fundamental 
characteristics of mem spectral analysis. The three most common view- 
points will now be discussed. The original viewpoint is that MEM is 
equivalent to maximizing the average spectral entropy of a Gaussian, band- 
limited time series subject to the constraint that the estimated power 
spectral density and the estimated autocorrelation function form an exact 
Fourier transform pair (Chen and Stegen, 1974; Newman, 1977). This can 
also be interpreted as determining an optimum power spectral density func- 
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tion that keeps the entropy of the empirical time series stationary with 
respect to the selection of extrapolated values of the estimated autocor- 
relation function (Radoski et al., 1975). 

A second viewpoint is that mem is equivalent to an "optimum" method 
for extrapolating a truncated autocovariance function (Veltman et al., 1972). 
The criterion for "optimum" is that the entropy of the time series model 
must be maximized at each step of the extrapolation. Van den Bos (1971) 
has demonstrated that this criterion is equivalent to fitting the data with 
a finite autoregressive process model with a least-square-error criterion. 
This is sometimes referred to in the literature as fitting an all-pole 
model to the data. This nomenclature comes from the z-Transform descrip- 
tion of the transfer function used to model the power spectrum (Stanley, 
1975). This type of estimator contains only poles (no zeros except at the 
origin) in the z-plane. 

The third viewpoint is that MEM is equivalent to modeling the desired 
spectrum with the output of a filter driven with white noise. The empiri- 
cal data function is processed to obtain the estimated filter spectral 
response. The filter has been variously termed a prewhitening filter, a 
Weiner prediction filter, and a least-squares inverse filter (see Chapter 
XIII). An equivalent mathematical model for the empirical data function 
is the convoltuion of a finite autoregressive series and a white noise 
series (Radoski et al., 1975). In this type of processing the input time 
series is converted to a white noise sequence (See Equation 14.24a) and 
the prewhitening filter is used to characterize the spectrum (Ulrych, 1972b). 
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A fourth viewpoint, which is actually a combination of the others, is 
that the entropy loss of the filter should be maximized subject to the con- 
straint that the inverse transform of the MEM spectral estimate yield the 
exact autocorrelation values. As with the third viewpoint, a filter is 
found that can generate the correct autocorrelation function when driven 
with white noise but it must also reduce the entropy by a maximal amount. 
This viewpoint emphasizes the First Principle of Data Reduction (Chapter 
VII) because it minimizes the biasing effects of missing data. 

C. Entropy Definitions for MEM 

The concept of entropy is used in many scientific disciplines. Entropy 
as a thermodynamic state function in statistical mechanics is a measure of 
the randomness of the system. Entropy quantifies the lack of knowledge 
about the exact state of the system at any instant in time. In information 
theory (Beckmann, 1967) the entropy of a source of information is defined 
as the average self-information per state of the system. In communication 
systems the entropy is often called the average information rate in bits- 
per-symbol. The entropy of a signal will be maximum when all states or 
symbols are equally likely. Two concepts that will be used in MEM deriva- 
tions are those of a entropy power and entropy loss in a linear filter. 
Mathematical definitions will now be given for the various entropy quanti- 
ti es. 

The entropy of a discrete system of N states, E , with probabilities, 

P , is given by: 

N 

H= - 2 p n ta(p n ) 
n=l n Q 


(ISC.I) 
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If H is maximized subject to the constraint that the sum of all probabili- 
ties is equal to unity, the distribution of probabilities that yields max- 
imum entropy is given by (Beckmann, 1967): 

P n=l,2, 3, ••• ,N (15C.2) 

n N 

A convenient definition for the entropy of a continuous distribution 
of a single variable is (Goldman, 1953; Beckmann, 1967): 

H= - Sp(x)£n(p(x))dx (15C.3) 

If H is maximized subject to the constraint that the variance of x remains 
constant, the calculus of variations solution gives a Gaussian distribution. 
The entropy of a Gaussian distribution is given by: 

H = -tn(2Wp % (15C.4) 

It is also true that, for a constant entropy, a Gaussian distribution has 
the smallest variance of all possible distributions. 

An excellent discussion on the effect of a linear transformation on 
entropy has been given by Goldman (1953). In essence Goldman shows that 
the entropy of a signal vector y^ produced by a linear transformation of 
the vector x^ is given by 

H =H +ta\a.. 1 (15C.5) 

y x 1 ij 

where la.. I is the determinant of the transformation matrix defined by: 
ij 

N 


(15C.6) 
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Random noise may be characterized by sampling points in the time domain 
which are statistically independent and have distributions which are Gaus- 
sian. The entropy per degree-of-freedom for random noise is 

h =£n(2necr 2 ) % (15C.1) 

n rr 

where o is the variance of the noise. To obtain the total entropy it is 

n 

necessary to multiply by the Nyquist frequency, f N =l/2At, and the dura- 
tion or observation period, TN ' of the signal: 


H 

n 


= 2f T h 
N N n 


The entropy power (Goldman, 1953) of a signal is: 


P 

e 


1 

2ne 


2h 

n 

e 


(15C.8) 


(15C.9) 


For Gaussian noise the entropy power is equal to the variance. Entropy 
power is always less than or equal to the time average power. 

Entropy loss in a linear filter is a function only of the frequency 
response of the filter and is most easily described in the frequency domain. 
The derivation for entropy change in a linear filter is analogous to that 
for a linear transformation since filtering is a linear transformation 
process. For N sampling points in the frequency domain the entropy at the 
output, Hy, is related to the entropy at the input, H x , using (Goldman, 

1953) 

N 

H = H + £ to(G |G(f )| ) (15C.10) 

yx. o' n 1 

J n=l 

where G q is the filter gain and G(f) is the normalized frequency response. 

In the limit as the frequency sampling interval goes to zero and the 
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frequency response becomes continuous, the entropy change becomes: 

^2 2 

H =H X + T N J to(G^|G(f)bdf (15C.1I) 

y o 

The entropy-per-degree-of-freedom can be obtained by dividing (15C.11) by 

2 W 

1 | f N 2 2 

h y =h x + ^ J ^(G o iG(f) 1 )df (15C.12) 

o 

The change in entropy is obtained both from the filter shape and the filter 
gain. The integral can be separated into: 



2 

to|G(f)j df + -ta(G o ) 


(15C.13) 


In most of the literature and for the rest of this text the filter gain is 
assumed to be normalized to unity and only the integral term is considered. 
The entropy power at the output of the filter is given by: 


P 

e 



(15C.14) 


If the class of power spectra to be analyzed by MEM is restricted to 
those that can be represented by a linear filter of finite length, we can 
define an average spectral ent-copy density or spectral entropy rate as: 

i +£ » 

h x*4|-J’ tc[5 x (f)]df (15C.15) 

_f N 

The power spectral density function, 5 (f), is two-sided and analogous to 

X 

I G (f ) 1^ • This is the definition that will be used for MEM (Lacoss, 1971). 
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D. Mathematical Bases for MEM 

A good mathematical description of maximum entropy spectral analysis 
can be presented using any or all of the viewpoints discussed in Section B. 
Three somewhat independent derivations will now be given. 

1. Maximum average spectral entropy basis 

The basis first proposed by Burg (unpublished 1967) maximizes the spec- 
tral entropy subject to the constraint that the inverse Fourier transform 
gives the exact values of the finite autocorrelation function. For a Gaus- 
sian process with a continuous bandlimited power spectrum, this entropy 
measure (Equation I5C.15; Smylie et al.,. 1973) is represented by the inte- 
gral: 

l +f * 

h =7 T-J* Ms <f)]df (15D.1) 

T* 

Restrictions on S v (f) are that it be nonzero in the Nyquist frequency in- 
terval and that it be representable as the power transfer function of a 
linear filter. S (f) is considered to be the "true" power spectral density 
of the process and ^ is the maximum frequency of nonzero spectral con- 
tent. This would be the folding frequency for a sampling system or the 
Nyquist frequency for a bandlimited signal. 

From previous theoretical considerations we know that the power spec- 
tral density function and the autocorrelation function form a Fourier 
transfonn pair as represented by the Wiener-Khinchine relations (Equation 
4A.9).From transform theory it follows that the inverse Fourier transform 
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of the continuous and band I i m i ted spectral density produces an autocorrela- 
tion function which may exist over the entire autocorrelation domain. It 
also follows that the autocorrelation function cannot be finite in extent 
because two finite functions cannot form an exact Fourier transform pair 
(uncertainty principle). If either the autocorrelation function or the 
spectral density are exactly known, the other may be computed using the 
transformation: 

J-Hs <f)}=r S v (f)e+i 2rrfT df=R v (T) (15D.2) 

X * 

A capital R is now used to represent autocorrelation but it has the same 
meaning as small r in Chapter XIII. 

Of course for practical problems the data function is finite and only 
an estimate of the true power spectrum can be obtained. A finite data 
function produces an estimated autocorrelation that is both truncated arid 
statistically variable. Both of these important effects must be accounted 
for in a practical MEM spectral estimator. As a start we will first con- 
sider the problem of truncation. 

To be able to satisfy Equation I5D.2 the truncated autocorrelation 
function must be extrapolated in some optimum manner. To examine this 
possibility the autocorrelation function is separated into a "known" com- 
ponent, ft (T), defined in the interval ITUT and an "unknown" component 
x m 

r x (T) representing the autocorrelation for all lags IT l>T. The inverse 
transform relationship of (15D.2) can now be separated into the two com- 
ponents: 
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r a cf)e + ; 2 " £T df=fe , t|st 

x 1 1 m 


J ^ x (f)e +j2TTfT df = R x (T) | t | > T 

_f N 


(15D.3) 


(15D.4) 


t will be defined as the maximum lag at which values for R (T) are 

(II A 

known. The so-called "known" component plays a dual role in MEM analysis. 
For developing the theory it is assumed to be a truncated version of the 
exact or "true" autocorrelation function while in actual application it is 
an estimate based on empirical data. For the analysis to follow it will 
be assumed to be exact while in Sections F and G i t will be an estimate. 

When the maximum entropy estimate satisfying (15D.3) is obtained, the 
"unknown" component given by (15D.4) is completely specified, it can be 
referred to as a maximum entropy extrapolation of the autocorrelation func- 
tion. From this viewpoint the maximum entropy criterion is equivalent to 
the a priori assumption that the "best" spectral estimate must be the most 
random of all possible estimates that is still consistent with the "known" 
(but truncated) autocorrelation function. This also means that the extra- 
polated part of the autocorrelation function adds the least amount of "in- 
formation" to the spectrum. 

The derivation that now follows is based on the calculus of variations 
material presented in Part J of Appendix I. The entropy measure that will 
be maximized is given in Equation 15D.I and the constraint equation will be 
a sampled version of (15D.3). The extra degrees-of-freedom which allow h x 
to be maximized are represented by Equation I5D.4. 
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First, the use of a sampled version of Equation I5D.3 must be justified. 
Now because we are interested only in S(f) defined in the interval tfl s; f 
it is possible to represent S(f) either by Equation 15D.2 or as a periodic 
function with period 2f^. The mathematical restriction imposed by this 
representation is that we never try to use it for |f|>f^. The fact that 
a continuous and periodic spectrum can be represented by an infinite sum 
of sampled autocorrelation values (Stanley, 1975) leads to the selection 
of the following infinite series and basis set for S(f): 

s ‘ <£) = R x (ki « e ' j2n£klT <f N ? 2a y dSD.S) 


The sampling period in the autocorrelation domain is related to the spectral 
width by At = 1/2 • To verify that this basis is valid, it is substituted 
into the following sampled version of (15D.2): 

+f N 

I S (f)e +j2T7 ^ nAT df = R (nAT) (15D.6) 

x x 


After substitution, integration and summation can be interchanged to give 
the form: 


+« 

At S R (kAT) 
k=-» x 



+j2rrf (n . k) Ardf =R 


x 


(nAT) 


(15D.7) 


To show that this is an identity, the form of the integral is compared with 
the representation of the Kronecker delta function given in Equation A2.33. 
This representation is obtained by identifying £ n = 1/2At and T =2f^ to give: 

At SV J 2TTf(n ' k)AT df = 6 

_f nk 


(15D.8) 
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The identity in Equation I5D.7 becomes 

+eo 

2 R x ( kAT)6 ak = R x (nAT) 


(15D.9) 


and we have shown the representation in (15D.5) to be valid. 

A corresponding sampled representation for Equation I5D.3 is obtained 
by defining a maximum known lag index m = t / At. This gives 2M + 1 known 
lags to define the bandlimited spectrum. The constraint equation and cor- 
responding series definition for £(f) become: 


J § ( f) e+j2nfn4T lf= A (n4T ) 

-S. * 


[n= 0, +1, +2, ••• , +M} 


(15D.10) 


+M 


l(£)=4TE & (Ur)e'j 2nfkiT 

x k=M x 


(15D.H 


From the calculus-of-variations solution and Equations I5D.I and 
I5D.IO we can identify the following terms for the E uler-L agrange equation: 

F(fJ(f)) = (f)] (15D.12) 

x 

Q n (f,S x (f» = ^ x (f)e +j2TTfnAT (15D.13) 


The E uler-L agrange equation is: 


. 3 Cf(M (f» - 2 X (n)Q (f,^ (f»] = 0 
(f) x n— M x n x 

X 


(15D.14) 
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The partial derivatives are 


3F = l 

* ’ 4 W f > 


= p +j2nfnAT 

as 


and the equation is evaluated to be: 

+M 

1 £ \ (n)e + j 2rTfn ^ T = 0 

4f A (f) n=_M x 


(ISD.IS) 


(ISD.16) 


From this solution we see that both the Lagrange multipliers and the inverse 
spectrum must be finite. This reaffirms the constraint of nonzero Sff). on 
MEM spectral estimates. 

The next step in the solution is to solve' the Euler-L agrange equation 
for § (f) in terms of the Lagrangian multipliers, X (n), and frequency, f. 

A A 

This solution is substituted into the constraint equation and the 2 M +1 
values of the autocorrelation function are used to solve for the 2M + 1 
values of X (n), the Lagrange multipliers. The solution is completed when 

X 

A 

the X (n) are used in the explicit equation for 5 (f). The explicit equa- 

X X 

tion is obtained by rearranging (ISD.16): 

£ (0 =1 [ £ X (n)e +j2TTfnAT ] (ISD.17) 

x n=-M x 


It is pointed out at this time that Equation 15D.16 is a finite and sampled 
version of Equation ISE.6. 

The direct method of solving ISD.17 is to use a Laurent series expan- 
sion and equate it term-far-term to Equation ISD.II. An alternate method 
which derives the spectral estimate directly in terms of the autocorrela- 
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tion coefficients used in the Burg algorithm (Section G) will be presented. 
The derivation essentially follows one given by Ulrych in Smylie et al.\ 
(1973). 

The z-transforms of Equations ISD.17 and ISD.S are equated to give: 
ATR(z) = 4 ^ (7) 23e -j2nfAT (ISD.IS) 

The polynomial, A (z ) , represents the double-sided z-transform of the series 
in Equation ISD.17 and, as such, is an ideal (not estimated) representation 
of the Lagrange multipliers. Conceptually, R(z) is also the infinitely 
long Laurent series expansion of l/A(z). If the z-transform of (.15D.11) is 
represented by atR(z), then R(z) and &(z) must have equal coefficients for 
like powers of z. We also know that \ x (n) =\ (-n) and R x (nAT) =R x (-nAT) 
because ^ x (f) must be a real function. Since these coefficients are Hermi- 
tian, the polynomials R(z) and A(z) can be factored into the product of two 
polynomials and Equation ISD.IS can now be written as (Smylie et al, 1973): 

A , tC(z)C*( | i z *) = l *_ * (ISD.19) 

^f N Q(z)Q (liz ) 

The new polynomials are defined by: 

R (z) =C(z)C*(liz*) (ISD.20) 

A (z) =Q(z)Q*(l iz*) (ISD.2I) 

The symbol Q for the polynomial must not be confused with that for the 
generalized function in the E uler-L agrange equation of (ISD.13). A similar 
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process for obtaining a polynomial product is the spectral factorization 
for rational spectral densities discussed in standard texts on signal anal- 
ysis (Cooper and M cGillem, 1971). 

The system of equations which represent the maximum entropy solution, 
for example Equation 13.48 in matrix form, can be written in series form as 
K A 

£ a..(n)R (k+n) = -pj (k) (15D.22) 

k=0 K X K o 


where a } ,(0) =-l and S Q (k) is the Kronecker delta function. The mean-square 

error of the estimate will be represented by which is equivalent to the 

. . . A 2 
notation a . 

K. 

We now implement a mathematical technique that will help simplify the 
solution. The system of equations represented by (15D.22) is augmented with 
the arbitrary series 


h(k) =0 [k=0 , 1,2, — , K} 

#0 [k= -1, -2, -3, •••, -K} 

to give: 

K 

£ a K (n) R x (k + n) = -P K 6 o (k) - h(k) 

{k=Q,+l,±2, 

The z-transform of (15D.24) is the start of the solution: 
A(z)ft(z) =P k +H(z) 

A(Z) =l-a K (l)z- a K (2)z 2 - ,..-a x (K) Z K 


(15D.23) 


(15D.24) 


(15D.25) 

(15D.26) 


H(z) =h(-l)z-l+h(-2)z-2+ ••• +h(-K)z-K 


(15D.27) 
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A (z) is the z-transform of the digital filter (Chapter XIV) characterizing 
the autoregressive process. 

Since ft (z) also consists of Hermitian coefficients it likewise can 
be factorized into the special form 

"k "k 

ft(z) =U(z)U ( 1 1 z) =U(z)V(z)/z K (15D.28) 

where the product polynomials are defined by: 

U(z) = u +UjZ + ... +u r z K (15D.29) 

V(z) = u q z K + u^z K ^ + . . . + (15D.30) 


Substituting (15D.28) into (15D.25) and dividing gives the desired result: 

K 


_ PKz H (z)zN 
A(z)U(z) 'V(z) + V (z) 


(15D.31) 


The rather elaborate means by which we have arrived at Equation I5D.31 are 
necessary so that the concluding steps, including a polynomial division, 
are easier. 

Typical terms on the left-hand side of (15D.31) are: 


A (z)U (z) = u Q + CUI - a K (l)u o >+ .•• + [i^ - u^OO JzK 
- [u^K - 1) +a K (K)u K _ 1 > 2K “ 1 - n^Wz® 


(I5D.32) 


This is a polynomial of degree +2K. Typical terms for the polynomial 
divisions on the right hand side are: 


K 

PKz + H (z)zK 
V (z) V (z) 


P K h -l 

[-§+ (-i 1 




u 


Ul N -1* 
~2 )z + 
u 

o 


. + C) 2 - K ] 


(15D.33) 
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Since the exponents on the left-hand side are positive and those on 
the right-hand are negative, equating equal powers of z on both sides of 
Equation I5D.3I gives the trivial result: 


u 


o 



(ISD.34) 


The coefficients of all other powers of z are zero so ( I S D .31) reduces to 
the result: 


A (z)U (z) u Q 



* 

u 

o 


(15D.35) 


In a similar manner we can also obtain the result: 


k k k k 

A (liz )U (liz) 



(15D.36) 


Combining these results and relating them to (15D.28) gives the z-transform 
of R x (k£T) in terms of the z-transform of the autoregressive series: 

ft'(Z) =U(z)U*(liz*) = P * — (15D.37) 

A (z)A (liz) 


Substituting this result into Equation ISD.II and evaluating the z- 
transform on the unit circle between 0 and tt gives the maximum entropy 
estimate of the power spectral density function: 



atp k 

A( e ‘- i2lTfAT )A*(e“ j2TrfAT ) 


(15D.38) 


From Equation I5D.26 we get the identity 
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A (e- jZtTf&Tj = | _ | a(k)e _j2tTfkAT 
k=l ^ 


(15D.39) 


and Equation I5D.38 can be put in the usual form given by Equation 15F.I. 

In the cause of academic curiosity we can determine the Lagrange multipliers 
from 


A(Z) = 2i A (Z)A 
K 


* * 
(hz ) 


(15D.40) 


where: 


K 

A(Z) = E X (n)Z~ n (15D.4I) 

n=-K x 


The solution is obtained by equating like powers of Z in the equations: 


+K K K 

E \(n)z" n =[l- E * (k)Z k ][l- Z a^Z - *] (15D.42) 

n=-K X k=l ^ k=l K 


This completes the derivation of the maximum entropy spectral estimate. 
Alternate derivations are given in' the next two parts of this section. 

2. A utocorrelation functi on extrapolation 

The following description of the MEM spectral estimator is based on 
the desirable property that the autocovariance matrix for a real finite 
autoregressive process must be semipositive definite (Veltman et al., 1972). 
This implies that the determinant of the autocovariance matrix must be non- 
negative. This condition guarantees that the resulting estimated power 
spectral density function will be nonnegative over the interval Ifls^. 

The analysis using the autocovariance matrix is based on the fact that it 
can be used to obtain the estimated spectrum by applying the Fourier trans- 
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form directly or by generating the autoregressive coefficients (Chapter XIV) 
and using the autoregressive spectral estimator. 

As in the previous section we will assume that the first 2N + 1 auto- 
correlations are known exactly and that the process is stationary and 
Gaussian. The maximum entropy extrapolation problem is to find the (N +i) 
autocovariance value by maximizing the entropy defined by the autocovari- 
ance matrix. 

To simplify the matrix notation for this section we will use the ab- 
breviation r(n) =R x (nir). The autocovariance matrix of order N will be 
represented by R(N) where the matrix elements are defined by: 


r.. =r(j - i) 


(ISD.43) 


The autocovariance matrix for an (N-M)-th order autoregressive process 
is given by: 



reO) 

r(i) • • 

• r(N ) 

R(N) = 

r(-l) 

reO) 

r(N - 1 


r(-N ) 

r ( - N + 1) 

. reO) 

For real processes 

the correlation 

function 


(ISD.44) 


would be possible to rewrite the matrix with all positive subscripts. To 
emphasize the fact that the function is double-sided we will keep the nega- 
tive notation for the moment. 

The autocovariance matrix, R(N), is extrapolated to one order higher, 


R(N +1), by maximizing the entropy measure defined by (van den Bos, 1971 
and Smylie et al ., 1973): 
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h =^a{(2rre) N+2 detCR(N+D]}^ (15D.45) 

This is the entropy of a N + 2 dimensional Gaussian probability density 
function with covariance matrix R(l\l + 1). To maximize h with respect to the 
unknown correlation, r(N +1), the first derivative is set to zero: 


5h \ _ $det f R ( N +1)1 = 0 

3r(N + i ) (2ne)F+2 det [R(N + l)] Sr(N + l) 


(15D.46) 


This is equivalent to finding r(N + 1) such that: 


Bdet LMK±!)1_ = 0 

3r(K+ 1) 


(15D.47) 


From Equation 15D.46 we see that R(N + 1) must be positive definite so that 
det[R(N+l)] is positive and nonzero. The derivative of det[Ft(N +1)J is 
obtained by first reducing R(N + 1) to the next lower order (this is the 
matrix minor of element R^ +2 N+2 > and then applying a lowering operator to 
all i ndi ces. This gives: 


5det[R(N + l)J 

3r(N+l) aet 


r(-l) 

reO) 

r(N-l) 

r ( - 2 ) 

r ( - 1 ) . 

r(N -2) 

■ 

r(-N-l) 

■ ■ ■ ■ 

r(-N ) 

■ ■ ■ ■ 

r ( - 1 ) 


= 0 (15D.48) 


Two additional useful properties of det[R(N+l)] can also be demonstrated. 
The first is that it is a quadratic function of r(N + 1): 


det[R(N + 1) ] =<* + a. r(N + 1) + a,r 2 (N + 1) 

O t L 


(15D.49) 


This is obtai ned by actual ly expanding det[R (N + 1) ] , setti ng r(-m) = r(m) , 
and collecting terms. The coefficients o^, o^, and a 2 are functions only 
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of the elements of R(N). The second property is that the second derivative 
of d et [ R ( N + 1) ] is always negative 


S det[R (N + I ) L = -2 det[R (N - 1)] 
Sr 2 (N+ 1) 


(15D.50) 


because det[R(N - 1)]>0. From the quadratic representation we can immedi- 
ately identify the coefficient to be: 


<y 2 = -det[R (N - 1)] 


(I5D.51) 


The quadratic now takes the form: 

det[R(N+ 1) ] = Qr Q + Q' 1 r(N+ 1) - |a 2 |r 2 (N+l) (I5D.52) 

A typical plot of det[R(N + 1)] as a function of r(N + 1) is a parabola that 
is concave down. A valid solution can exist only in the region det[R(N + 1) ] 
> 0. The parabolic form also guarantees that there is only one maximum in 
the valid region. Since the ordinate values of the parabola are symmetrical 
about the maximum, it also follows that maximizing the entropy is equivalent 
to choosi ng a value for r(N + 1) that I i es at the midpoint (V eltman et a I 
1972) of the region where detLR(N +l)]>0. 

The final solution for the unknown autocorrelation value is obtained 
by differentiating Equation 15D.52 and solving for r(N+l): 


Sdet [R(N + 1)L = ~ 2L I (N + 1) =0 

“^r(N4^) a l~ a 2 r 

°1 

r(l\l + !) = 2 |£ ~ " f ' = 2 det t-R-(N-l-)-=l 


(15D.53) 


(150.54) 
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A satisfactory solution requires that be nonzero otherwise the extra- 
polation would terminate. The determination of requires several matrix 
manipulations starting with Equation 15D.48. To eliminate notation com- 
plications caused by negative indices the even property of r(n) is invoked 
to give: 


r(l) 

r(O) 

. r(N-l) 

r ( 2 ) 

r(l ) 

r(N -2) 

r(N) 

i ■ ■ ■ 

r(N-l) 

• • r(O) 

r(N +1) 

r(N ) 

--- r( 1 ) 


(15D.55) 


Expanding the determinant using the last row and the method of minors 
produces the result: 

N- 

r(N + 1) det [R(N - 1) ] + Z r(k) (-l) N+k+1 w , k =0 (15D.56) 

k=l w ‘ 1. 

The m n+1 k are the corresponding matrix minors of the elements in the bot- 
tom row. Finally we can identify the coefficient as: 

a ! = \% r(k)( - |)Ntt+ V, k < 15D ' 57 > 

In conclusion, the HEM extrapolation algorithm is simply a step-by- 
step computation of higher order autocorrelations to achieve an arbitrarily 
high order matrix. The matrix manipulations indicated by Equation I5D.57 
are not implemented in practical algorithms because of the computational 
effort involved. In the next section an equivalent estimate based on an 
autoregressive model is derived that has better computational efficiency. 
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3. A utoregressive or predictive-fi Iter basis 

The third mathematical description of MEM involves the definition and 
calculation of a prewhitening or predictive filter. In Chapter XIII it was 
shown that predictive filtering and autoregression are equivalent viewpoints. 
Before starting a detailed analysis it will be helpful to briefly review 
some basic system analysis concepts that will help tie together the equiva- 
lent viewpoints. 

A continuous-time, linear, time-invariant system with a single input 
x(t) and a single output y(t) is illustrated in Figure I5D.I. The differ- 
ential equation which describes the system is given by: 


M M-l 




+ 


•• + a y 

0 


K 

dx 


K + b 


K-l 

dx 


dt 


K-l 


dt*' 1 


+ 


'. + b 0 x 


(15D.58) 


x(t) 



y(t) 


Figure 1 5 D - i . Continuous-time, Linear, 

Time-invariant System. 

An equivalent discrete-time system, is likewise described by a linear 
difference equation (Stanley, 1975) with constant coefficients as given by: 

a Q y(ii) + a ][ y(n - 1) + a 2 y(n - 2) + . . . + a^fn - M) = 

b x(n) +b,x(n- 1) + ... +b x(n- k) 

O 1 K. 


(ISD.59) 
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The z-transform definition for the transfer function of a discrete-time 
system is obtained from (ISD.S9) by applying a unit impulse: 

b + b Z -1 + b Z _2 + ... +b z" K 

H(Z) = -2 ^ ^ (ISD.60) 

a Q + a^Z + + . . . + a^Z 

A special form of this transfer function is used in MEM spectral analysis 
and is called the "all-pole", model. The terminology "all-pole" model (Van 
den Bos, 1971) in the literature refers to a transfer function that has no 
zeros in the z-plane. As a consequence, the numerator polynomial in Equa- 
tion ISD.60 has all zero coefficients except for b Q • 

If the input to the filter is white noise we say the output time series, 
yet), was generated by a moving average process (See Chapter XIV), Equations 
14.6 and 14.19). The output time series power spectral density is completely 
characterized by the coefficients, b • 

If the input to the filter generates an output which is white noise 
we refer to the input time series as representing a finite autoregressive 
process of order M. The power spectral density function is completely 
specified by the coefficients, a n (Equation 14.40). The "all-pole" model 
is represented by the finite autoregressive series given in Equation 14.24. 

The Yule-Walker equations are used to relate the finite autoregression 
coefficients, a n , and the stationary autocovariances. These relationships 
are presented in detail in Equation 14.42. This linear set of equations 
is expressed in matrix form in Equation 14.46. After manipulating the ma- 
trix form, changing the notation so C(n)=r(a), and adding a row of order 
N + l, the Yule-Walker equations can be. expressed as: 
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r ( 1 ) 

r(O) 

r(N-l) 

r ( 2 ) 

r(l) 

r(N -2) 

r(N +1) 

r(N) 

r ( 1 ) 



1 


0 




0 




0 


(ISD.61) 


This is a homogeneous system of N+l linear equations and from Cramers 
rule the characteristic determinant of the system must be zero; therefore: 


det 


r ( 1 ) 

r(O) 

r(N-l) 

r(2) 

r ( 1 ) 

r(N -2) 

r(N +1) 

r(N) 

r ( 1 ) 


= 0 


(ISD.62) 


Since this solution is identical to that obtained by maximizing the entropy 
(Equation 15D.S5) the autoregressive model is equivalent when the autocorre- 
lations are known exactly. If it is desirably to obtain the extrapolated 
autocorrelation values using an autoregressive model, they are easily ob- 
tained from the following modification of Equation 14.42a: 

N 

r(N + 1) = £ a r(N + 1 - n) (15D.63) 

n=l n 


This requires a knowledge of exactly N values of a n and N values of r(k). 
The power spectrum obtained from the autoregressive coefficients is equal 
to that obtained by a Fourier transform of the autocorrelations. The auto- 
regressive technique is superior to the extrapolation method in the last 
section because the coefficients, a , are easily obtained from the known 
autocorrelations by using Equation 14.47. The practical implementation of 
an mem spectral estimator is presented in Sections F and G. 
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E. The Companion Function for R (T) 

At the beginning of the author's research, one of the first problems 
attempted was to relate the maximum entropy viewpoint of spectral analysis 
to the W iener-K hinchine relations (Equations 4A.8 and 4A.9). The concepts 
of mem were not well-understood and it was hoped that this approach would 
provide the needed insight. The result of this effort was the development 
of the generalized calculus-of-variations formulation for integral con- 
straints given in Part j of Appendix I . The maximum entropy measure dis- 
cussed in the previous section and its integral constraint equation are an 
application of the calculus-of-variations solution that was derived. 

This approach to the investigation of the properties of MEM led to the 
discovery of a companion function, X (T), for the autocorrelation function, 

R (T), which for a certain class of processes gives an equivalent descrip- 

X 

tion of the power spectrum of the process. The results that follow will 
seem peculiar because the constraint equation completely specifies S (f) 

X 

and thus allows no degrees of freedom. The reasons for presenting this un- 
usual approach are to show that MEM is consistent with the W iener-K hinechine 
relations for specific classes of spectral functions and to show the dis- 
covery of the companion function. 

The entropy measure to be maximized is given in Equation I5D.I and the 
integral constraint equation is given by the Fourier transform relationship: 

M s (Oe+j^Tjf =R (T) (15E.I) 

_C0 X X 

Comparing Equations I5D.1 and I5E.I with Equations AI.72 and A1.73 in Part J 
of Appendix I we can identify: 
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/ 

F(f,S x (f)) = -ta[S x (f)J 

Q(T,f,S x (f)) = S x (f) e + ^ 2TTfT 

The factor (4£ N )’ 1 will be neglected in this section. 

The appropriate E uler-L agrange equation is (Equation AI.83): 


3F 

3S 


+® 


x (T) f dT - 0 


(ISE.2) 

(I5E.3) 


(I5E.4) 


Evaluating the partial derivatives in (15E.4) and solving for S (f) in terms 
of X (t) and f gives the solution: 


SF = 1 +j2nf t 

3S S x (f) 3S 


(ISE.5) 








\ (T)e +J2,lfT dT 
X ' 


0 


(ISE.6) 


Th: 

conjugate of the Fourier transform of the continuous Lagrangian multiplier, 
X (t), is equal to the reciprocal of the power spectral density function 
for the process: 

" 5E7) 

X 

This representation is restricted to processes which have a well-behaved 
reciprocal spectral density. The spectral density, must be continuous and 
nonzero for all f. This limits the representation to stochastic processes 
when the frequency is not bandl i m i ted . For band I i m i ted processes the dis- 
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cussion in Section D applies and the reciprocal spectral density is always 
well behaved. 

Continuing with the calculus-of-variations method, the solution for 
S (f) is substituted into the constraint equation to give the relationship 
between \ x (t) and F^(T): 

/“ L e +J2n£T df = R (T) (ISE.8) 

-» J* (X x (T)} 

This integral equation can be solved for X (t) as a function of R (t) by 

X X 

taking the direct Fourier transform of both sides. This procedure gives 
the following results: 

1 =°F(R (T)fJ*{X (t)} (ISE.9) 

X X 

v t > - < |se - |o » 


Finally the solution for J^(t) is used in (ISE.G) to derive equations for 
S (f) as a function of X (t) and R^(T): 


S (f) 

X 


< T , TX< t » 


-'Tm (T)j 
X 


(ISE.il) 


The power spectral density function, S v (f), is specified completely by the 
Fourier transform of the autocorrelation function (W iener-K hinchine rela- 
tions) or alternately by the reciprocal of the complex conjugate of the 
Fourier transform of the continuous Lagrangian multiplier function, X (T). 
Because of this analogy, the function X (t) will be called a companion 

X 

function for the autocorrelation function, R (T). 
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After a considerable amount of effort was expended on trying to deter- 
mine an interpretation for the companion function, a simple solution was 
seen that now will appear trivially obvious. Consider Equation I5E.9 for 
the case of a "real" process. The autocorrelation function is real and even 
and so is the companion function. The equation can then be rewritten as 


1 =°?{s x (t)}TC\(T)} (15E.12) 

or 

1 S (f ) A (f ) (15E.13) 

X X 


where A (f) is the spectral function represented by the Fourier transform 

X 

of \ (T). Comparing Equation 1 5 E .12 with Equation 5B.8 we can make the 

X 

following interpretations: 

1. Equation 1 5 E .12 represents a linear filter operating on a stochastic 

input to produce a white noise output. In short: an ideal- pre- 

whitening f i Iter. 

2. S (f) and R (T) represent the filter input. 

X X 

3. /^(f) and \ x (f) represent the filter transfer function where 
A x (f) ='R(f) ,2. 

4. The output spectral density function represents white noise with 
unity amplitude, S Y (f) = I . 

The Fourier transform relationships in (15E.12) can be replaced by the fol- 
lowing convolution integral: 


6(t) = 



-03 


R x (t)\ x (T- t)dt 


(I5E.14) 


This is yet another way of representing the ideal prewhitening filter. 
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Equation 1 5 E .14 is analogous to (14.21) for a finite autoregressive process. 
Also compare (ISE.13) and (14.25). 

In summary we can say that the companion function, \ (.,..), represents 

X 

an ideal linear prewhitening filter and that the filter transfer function 
is specified by: 

|H(f)| 2 (ISE.IS) 

F. MEM for Discrete and Finite Data 

For empirical data records the data function is finite and the esti- 
mated autocorrelation function is also discrete and finite. Furthermore 
the autocorrelation function is also statistically variable with the vari- 
ability increasing with lag. The exact autocorrelations used in the pre- 
vious sections will now have to be replaced with estimates. The second 
factor that enters the picture is the selection of the order of the process 
used to do the modeling. The order affects the resolution of the estimator 
and the numerical stability of the computer algorithm. 

Practical methods of implementing the MEM spectral estimator use the 
autoregressive model and compute the coefficients directly. This approach 
has the simplest numerical implementation because it can be done recursively 
and avoids the matrix manipulation needed in other approaches. This direct 
approach is the only practical one for large orders. The presentation to 
follow will show the methods employed to control statistical variability 
and to select a suitable order. 

To begin we will define all of the terms used in the autoregressive 
spectral estimator given by (Equation 14.52) 
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V« = 


■ kl V">* 


- j2rrfkAT| 


(ISF.l) 


where: 

f = frequency at which the spectral density is estimated, f is 
limited to the range, 

f = the Nyquist frequency or folding frequency determined by the 
sampling rate, ^=1/2 at. It will be tacitly assumed that the 
data function has been suitably preprocessed to eliminate aliasing. 

At = sampling interval or lag interval between samples in the autocor- 
relation domain. The lag interval is also equal to the sampling 
interval in the time domain, at = At. 

K = the order of the autoregression used to model the process. For 
most applications K is never larger than approximately one-third 
of the total number of data function samples. 

^(k) = {a^(k) : k= 1,2, •••, K} the estimated autoregressive coefficients 
used to model the empirical data function. A boundary condition 
for physical realizability is: a^QOcl. 

I (f) = Estimated power spectral density function at frequency f for a 

K-th order autoregressive approximation to the process. 

A 2 

Oj, = total mean-square error for a K-th order autoregressive model of 
the process. This is also the one-step prediction error given in 
Equation 13.36. It is also the estimated variance of the white 
noise sequence produced by removing the predictability in the data 
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function (Equations 14.43, 14.44, and 14.48). This is sometimes 
referred to as the total error power of the prediction. 

The system of equations used to represent the maximum entropy solution 
was discussed in Chapters XIII and XIV (Equations 13.48 and 14.47). This 
matrix formulation is repeated below in the notation that will be used in 
this section: 

r(0) r(l) r(K) 

r(l) r(O) r(K-l) 


?(K) r(K-l). • reO) 

We can now get down to the- crux of the problem encountered in the 
practical application of kem, autoregressive, or any other spectral esti- 
mation technique: 

How does one obtain "good" estimates of the autocorrela- 
tions, r(n), or the autoregressive coefficients, <u(k), 
directly from the finite data function? 

To begin, consider estimating the autocorrelation function: 

r(n) =E (x(t)x(t-n)} (15F.3) 

For a finite data function defined by {x(t): t =0, 1, 2, •••, N - 1 } an un- 
biased estimator is: 

. , N-lnl-1 

r(n) = K , 1 n [ J x(q)x(q+ Ini) 


■\' l > 

• a K (2) 




*2 

0 

0 


0 


(15F.2) 


(15F.4) 



251 


It is unbiased because for large N it converges to the theoretical values 
defined by Equation I5F.3. This estimator becomes unsatisfactor for large 
lags because the amount of the data record used is so small that the vari- 
ability of the estimate becomes intolerably large (Jenkins & Watts, 1968). 

A common solution to this that appears in much of the literature is to bias 
the estimator with a triangular window function (Schwartz and Shaw. 1975): 

x '(n)=(l - -W-)r(n) (ISF.5) 

This choice is adequate for some applications of spectral analysis and is a 
special case of the Blackman-Tukey method discussed previously. It is un- 
desirable in many applications because it causes smoothing in the estimated 
spectrum. The estimator in Equation I5F.4 may actually produce an FFT with 
negative components. In fact there is no assurance that the covariance 
matrix will be positive definite (Jenkins & W atts, 1968). 

We now proceed to demonstrate the computation of the autoregressive 
coefficients directly. An earlier paper by Levinson (1947) gives an excel- 
lent discussion on linear prediction and the least-squares prediction error 
■criterion for obtaining the "best" filter coefficients. J. P. Burg (1968) 
modified the Levinson algorithm to give an estimate that would guarantee 
that I a I ( I ) I would not exceed unity (Ulrych, 1972b). A brief presentation 
of linear filtering and prediction patterned after Levinson's treatment 
will now be given followed by a derivation of Burg's recursion method. 

Linear filtering and prediction can be briefly explained with the help 
of Figure 1 5 F - 1 . As much as possible, notation will be patterned after 
Chapters XIII and XIV. A known signal x(k) is corrupted by white noise §(k) 
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to generate a corrupted signal y(k). This corrupted signal represents the 
data function in the spectral estimation process. The data function is 
then "optimally" filtered by a convolution with h(n). The "best" estimate 
of x ( k) represented by x(k) appears at the output of the filter. The esti- 
mation error for the k-th sample is represented by e(k). The following 
equations apply 


y(k) 

= x(k) + e(k) 

(15F.6) 

e(k) 

•o< 

■ 

T< 

ii 

(15F.7) 


K 


x(k) 

= 2 h(n)y(k - n) 

(15F .8) 


n=0 


the 

order of the estimate. 




X(k) 


e(k) 


Figure 1 5 F - 1 . Linear Filtering and Prediction 

a measure of the quality of the estimate is the expected value of the mean- 
square error (Equation 13.36): 

2 K 2 
E {e (k)}=E {[X (k) - 2 h(n)y(k-a>r} (15F.9) 

n=0 


{k= 0, +1, +2, .... +»} 
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Denoting this error by M g and expanding the expected value using Equation 
15F.6 gives: 


m =E{x 2 (k)} - 2 £ h(n)E (x(k)x(k - n) } 
e n=0 

K 

- 2 £ h(n)E[x(k) ?(k - n)} 
n=0 

K K 

+ 2 2 h(n)h(m)E [y(k - n)y(k - m) } (15F.10) 

n=0 m=0 


By recognizing the appropriate autocorrelations and assuming x(k) and f(k) 
are totally uncorrelated we get: 

K 

M = R (0) - 2 2 h(n)R (n) 
ex n=0 x 

K K 

+ £ 2 h(n)h(m)R (m-n) (15F.11) 

n=0 m=o y 


It is important to note that R y (m- n) represents the autocorrelation func- 
tion of the generated or measured data. 

The best estimation in a least-square error sense for the filter coef- 
ficients is obtained when M g is minimized with respect to the h(n). The 
partial derivatives 


=0 [k=0, 1,2, •••, K} (15F.12) 


are evaluated to give 


K 

= -2 R (k) +2 £ h(n)R (k - n) = 0 
x n=0 y 


(15F.13) 
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which for a system of K + l equations: 

K 

E h(n)R (m - n) =R (m) m = 0, 1, •••,£ (15F.14) 

n=0 y ' 

The minimum error of the estimation is obtained by substitution of (15F.14) 
into the double summation of (15F.II) to give: 

K 

M (min) = R (0) - E h(n)R (n) (15F.IS) 

e x n=0 x 

The optimum filter gives a best estimate of the x(k) in a least-squares 
sense. The solution to the system of equations given by (15F.14) is pre- 
sented as an iterative technique in Levinson's paper and is referred to as 
the Levinson algorithm. This completes the presentation on linear filter- 
ing and prediction except to state that the infinite data model used would 
have to be modified for finite data before it could be applied to a prac- 
tical filter synthesis problem. 

The Burg recursion also determines the linear prediction-error filter 
coefficients and minimizes the mean-square error of the estimate. The ap- 
propriate system of equations for the Burg recursion given by Equation 
1 5 F -2 is analogous to Equation 1 5 F .14 augmented by Equation 1 5 F .15. It is 
interesting to note that Equations I5F.2 can also be written in series form 
(See Equation 150.22) as: 

£ atti)r(k+n) =r(k) - &(k) (k- 0, 1, 2, . •. , K} (15F.16) 

n=l k K ° 

The best linear one-step predictor in a least-square error sense is given 
by (Equation 13.35): 
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ft (t) = 2 - n) (15F.17) 


The one-step prediction error for a K-th order autoregressive model 
total mean-square error: 

4 = EfCx(t) - x (t) ] 2 } 

a 2 =E([x(t)-5 a (n)x(t- n) ] 2 } 

K n=l K 


is the 


(15F.18) 

(15F.19) 


For a finite data function defined by (x(n): n =0, 1, 2, •••, (N - 1) } 
Burg suggests defining both a forward prediction error and a backward pre- 
diction error: 

K 

ef(q) = x(q) - S ag(n)x(q - n) (15F.20) 

K 

e b (q- 1) = x(q - 1) - E a K (n)x(q + n) (151'.21) 

To eliminate end effects the predictor subscripts must be limited to 

forward: K^q^(N-l) (15F.22) 

backward: 1^qS(N- K - 1 ) (15F .23) 

inclusive: «£q £ (N - K - 1) (15F.24) 

where K is the order of the estimate and N >K is the total number of data 
samples. The mean-square error of a K-th order prediction would now be de- 
fined as the mean of the error for both forward and backward predictions. 
For N data samples: 
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, (N-K-l) - ~ 

4 ^(q) + £^(q-l)] (15F.25) 

The solution proceeds by minimizing Mj, with respect to the a^n) in a re- 
cursive manner and solving the system of equations for the a^n) . A pro- 
cedure for implementing the Burg recursion in a manner similar to the Lev- 
inson algorithm has been outlined by Anderson (1974). A derivation of this 
technique and a computer algorithm for implementation will be given in 
Section G. 


G. Burg mem Recursion Algorithm 

This algorithm determines estimates of the autoregressive coefficients 
a^fn) which are "best" in the following senses: 

1. The coefficients generate a maximum entropy spectral estimate when 
used i n Equation 15F .1. 

2. The entropy loss in a linear predictive filter is maximized. 

3. The estimated mean-square error of a K-th order prediction, c^, 
i s minimized. 

4. Equations I5F.2 are satisfied in a least-square error sense. 

5. la^K) I cannot exceed unity (Ulrych, 1972b). 

6. The autocovariance matrix is positive definite. 

7. No zero extension of the data is needed and no window function is 
used explicitly. 

For a finite data function defined by 


(x(t): t= 1,2,3, •••, N} 


(15G.I) 
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the average mean-square total prediction error for a K-th order process can 
be written as: 


1 N-K K 2 

= 2 ( N - K ) 2 [x(t)- 2 aK(k)x(t+k)l 

t=l k=l 


1 N-K 

+ 2{N _K) I [ x ( t+K 
t=l 


K 2 

2 a Y (k)x(t + K - k) ] 
k=l K 


(15G.2) 


The reader should convince himself that the equation above is identical in 
every respect to Equation 15F.25. The form and subscript notation now used 
are more convenient and closely conform to those used by Anderson (1974). 

The implementation of Equation 15G.2 by the minimization of is done 
with a recursion which starts with order K =1 and progresses to some maxi- 
mum value K =2, 3, "vK. This procedure generates a 2x2, 3x3, " 

K.m x K m autocovariance matrix which is both positive definite and maximizes 
the entropy. By generalizing this procedure it can be shown that the auto- 
regressive coefficients are related by (Smylie et al., 1973): 


v k) =v , 1 ( k ) - L (K - k) 


(15G.3) 


K =2, 3, •••, (N - 1) 
k= 1, 2, ( K - 1) 


A 2 

Using the notation, P K =°£> a recursion for P K is obtained by using Equation 
15G.3 in Equations 15F.2 or 15F.16. It is: 


P 

o 


K -i[1- V K) 1 


P„- 


1^2 A 

“ N tii x « = r(0) 


(15G.4) 


(15G.5) 
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At each step in the recursive process P K is the minimum value of at 
that step. As can be shown using (15G.3) the only degree of freedom at each 
new step is the coefficient^^). To obtain the least-squares solution, 
the average prediction error should be minimized with respect to a^K) , This 
gives the following system of equations: 


— — = n K =1 2 ••• K 

&L, 00" u K 


(15G.6) 


It will be seen shortly that the solution for ^(K) which minimizes is a 
function only of x(t) and can be calculated directly. 

Another important feature of the technique is that the estimated auto- 
correlation functions can also be obtained recursively (analogous to the 
Y ule-Walker equations) as 


m 

r(m) = S a (q)r(a-q) 

q=l 


(15G.7) 


where r(0) is given by Equation 15G.5. This is analogous to the system of 
equations represented by (14.42). 

A convenient formulation of the solutions represented by (15G.6) has 

been given by Andersen (1974) and will now be summarized. First define 

= -l so Mj, in Equation I5G.2 can be rewritten as: 

1 N-K k 2 

M*. =2(N-K) 2 C(- 2 ^(kMt + k) 

T t=l k=0 ^ 



K 

+ (- £ a_,(k)x(t + K - k» 
k=0 K 


2 


(15G.8) 


Next define ^(k) = 0, k^K in the recursion of Equation 15G.3 (except for 
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the term shown explicitly as ^(K) and substitute into (15G.8). With a 
change of sign inside the squared terms the results are easily determined 
to be 


\ 2 (n - k) 


+ (B K (t) ■ a K (K)B K (t))2] (15G.9) 


where: 


K 

B K (t) = 2 a . (k)x(t+ k) (15G.I0) 

k=0 


K 

B ’et) = S a. I ( K - k)x(t + K - k) (15G.II) 

K k=0 


t=l, 2, ••• , N - K 


which minimizes as defined by Equation I5G.6 is easily 
determined since B^t) and Bj/C t) are both independent of a^(K) . Taking the 
partial derivatives of Equation I5G.9 as described by Equation I5G.6, we 
obtain the desired result: 

N-K 

2 E B K (t)B K '(t) 

: < 15G - 12) 

S Bj(t) + B„ /Z (t) 
t=l 

A simple example of a first order autoregression will illustrate the dif- 
ference between estimating a^(K) with the Burg technique and using the 
estimated autocorrelations. The Burg technique gives: 


The value of 
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a 1 ( 1 ) 


N-l 

2 E x(t)x(t+ 1) 
t=1 


E Lx^ (t) +x^ {t+1 ) ] 
t=l 


The direct method gives: 


a^l) 



r(O) 


N-l 


N-l 

E x(t)x(t+ 1) 
t=l 


E x (t) 
t=l 


(1SG.13) 


(1SG.14) 


Both estimates are unbiased. The condition I^CK) 1<1 can be shown by ap- 
plying triangle inequalities to (15G.12). 

The final task that remains is to demonstrate useful recursion formulas 
for B K (t) and b'(c). Using (1SG.3) in (15G.IO) and (15G.1I) and paying 

Kb 

careful attention to the valid range of indices and previous definitions 
for outside the normal ranges gives: 





.jOC-UB^r) 

(1SG.IS) 

B^(t) 

= B K-l (t + 

1 ) - 


(15G.16) 


K = 2 , 

3 , 

-• K m 

(15G.17) 


t = 1 , 

2 , • 

»•*, (N - K) 

(1SG.18) 

Bj(t) 

= X t 


B^t) =x(t+l) 

(15G.19) 


This completes the presentation on the Burg mem Recursion Algorithm except 
for a few summary comments. 
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It should be strongly emphasized that the Burg recursion reduces but 
does not eliminate the effects of statistical variability in the estimated 
coefficients. Also the effect of numerical truncation in a computer imple- 
mentation of the technique may introduce zeros in the numerator of Equation 
I5F.I. The statistical variability of 
with K. This can be interpreted as a reduction in the amount of data used 
for the estimate. The reader can verify this by examining Equations I5G.9, 
I5G.IO, and I5G.II. The quality of the estimate of SK(f) as a function of 
K is best evaluated using both PK(or M^) and the variance of the estimate 
of P K * We know from previous work that e{p k ) = o k . These factors will be 
discussed more in Chapter XIX. 


, and consequently a^K), increases 
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XVI. ONE-BIT AUTOCORRELATION SPECTRUM 

Although the one-bit autocorrelation scheme of digital spectral analysis 
is well-known and widely used in radio astronomy (Weinreb, 1963), the method 
is not presented in most textbooks. This computationally simple digital 
technique makes use of the fact that only phase information is necessary for 
the determination of the shape of the power spectral density function of a 
Gaussian process. Amplitude information is necessary only for the deter- 
mination of the amplitude scale factor of the spectral density. 

To implement this technique, the data function, x(t) is processed by 
first sending it through a zero-crossing detector and then clipping the out- 
put to obtain digital logic levels. This process is described mathematical- 
ly as 


yet) = sgn(x(t) } (16.1) 

or: 

f +1 x(t) s 0 

yet) = < (16.2) 

1^-1 x (t) <0 

The digital signal, yet), is then sampled at a uniform clock rate to code 
the samples for digital processing. The resulting digitized time series 
can be used to obtain the estimated autocorrelation function of yet) by the 
following correlation process: 

Py(T) =-* fi y(k&t)y(kAt + T) (16.3) 

The estimate of the normalized autocorrelation function of the "undipped" 
signal is now obtained by the Van Vleck relation: 
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p (D = sin[.!!2 p (l 1 ) J (16.4) 

a y 

Finally, a normalized estimate of the power spectral density of the process 
is obtained by the Fourier transform: 

£ x (w) = k5(p x (t)} (16.5) 


The amplitude scale factor, K, may be found by measuring the power in the 
data function, x(t). 

A complete discussion of one-bit autocorrelation spectral analysis is 
given in Weinreb (1963). He derives the Van Vleck relation and also the 
mean and variance for the one-bit autocorrelation function estimate. He 
shows that, in terms of a large number of samples, N, the mean of the esti- 
mate is approximated by 

<Px>‘P*[ 1+ W ( v !»J (16 - 6 > 


and the variance by: 


*2 

a 

x 


— (1 _ 

4N K 


2 2 
Px^-P? 


(16.7) 


The hardware implementation of a simple one-bit crosscorrelator is 
relatively simple. The block diagram of a crosscorrelator designed by the 
author for radio astronomy research is shown in Figure 16-1. It is also 
possible to use the same system for autocorrelation if y(t) is a time-delayed 
version of x(t). 

For autocorrelation, it is simpler to provide the necessary time delay 
(or lag) by sampling the input signal and using digital delay techniques. 

The block diagram for a simple one-bit autocorrelator is shown in Figure 16-2. 
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One-bit autocorrelation techniques can be applied only when large 
amounts of data are available. The method offers simplicity in exchange 
for large data sets. Small data sets require the extrapolation and 
smoothing techniques inherent in methods such as MEM. 



Xl(t) 


x 2 (t) 
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Figure 16-1. One-Bit Crosscorrelator 
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Figure 16-2. Simple One-B it A utocorrelator 


266 







267 


XVII. DATA REDUCTION SCHEMES 

The various computer programs written for data reduction are described 
in this chapter. Each discussion includes an introduction, a list of pro- 
gram variables, and an example run. Programs 01, 02, 03, and 04 are in- 
tended primarily to be tutorial aids. Programs 05, 06, and 07 are user 
oriented and designed to be used to process data. For this reason, the last 
programs will be explained in much more detail. Listings of the programs 
are given in Appendix III while further examples of their use are given in 
various chapters. The printed outputs for the programs are also given in 
Appendix III and the reader must refer to theta for the examples. Each pro- 
gram number is keyed to an example in the appendix. 

A. PROGRAM 01 - Discrete Fourier Transform Testing 
1. I ntroduction 

This program was written to illustrate the digital Fourier transform 
and its inverse as derived in the text. The program consists of two sub- 
routines which can be used to illustrate various sampling effects on the 
estimates of the Fourier coefficients. The program is provided for user 
experimentation and as an example. It is not intended for routine data 
processi ng. 

SUBROUTINE FSRS computes an output time series corresponding to input 
Fourier coefficients. It is a program implementation of Equation 120.1. 

The program equations follow the more familiar Fourier series form given 
by Equation A2.2 in Appendix 1 1 . 
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SUBROUTINE LNSPTM computes the Fourier coeffi.c.ients from an input time 
series. It is a program implementation of Equation 12D.5. The program 
equations are the digital form of Equations A 2.4 and A 2.5 i n A ppendix 1 1 . 

2. I nput/output data defi niti ons 

A ( I ) = Fourier coefficients corresponding to the cosine inte- 

gral or even part of the input time series. 

B ( I ) = Fourier coefficients corresponding to the sine integral 

or odd part of the input time series. 
cmag(I) = Absolute magnitude of the i-th Fourier coefficient. 

CM agdb(I) = Absolute magnitude in dB. 

CPHASE(I) = Phase angle of the i-th Fourier coefficient referenced 
to the center of the input time series. 
dcv = Average or 'd c ' value of the input time series. 

F X ( I ) = Output time series generated by SUBROurINE FSRS. 

FVALUE(I) = Input time series for SUBROUTINE LNSPTM. 

NS = Total number of input samples of the time series (must 

be odd). 

TDEL = Equivalent sampling interval in seconds. 

I F = Fourier period, seconds. 

TN= Observation period, seconds. 

XDELTA = Sample times for the output time series. 

3. Example 

The input for the example program is the Fourier coefficients aj =4, 
b^ = 1 ■ The Fourier frequency is one H ertz and 21 samples are taken in an 
observation period of 3.1 seconds. 
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The program generates its own analysis data by using SUBROUTINE FSRS 
and the Fourier period ("true" period), Tp . 

The complex Fourier amplitudes are computed from these 21 samples 
using SUBROUTINE LNSPTM. The user must remember that this routine requires 
neither a sampling time nor an observation period. These are only used to 
interprete the output results. 

The last step is the computation of the estimated time series with a 
factor of four -increase in time resolution. This output can be compared to 
the "true" series to judge the quality of the estimate. 

The numerical results for this example are compared in Figure I2D-3. 
The quality and numerical stability of the estimate can often be 
judged by applying Parseval's theorem to the results. First, the complex 
amplitudes are determined using (A2.13): 

c =0 c, =2 c. =2 
o 1 -1 

1 , , 1 

c 3 2 J - c 3 ~ + _7 J 

Second, from Equation 12. A8, the sum of the squared amplitudes gives: 

+3 ~ _ 

R (0 ) = E | c 1= 8.50 volts^ 

n=-3 n 

From the estimated time series, the result approximated by (12B.29) is 

2N+I 

P = 2N + 1 kl ^=-8.70 volts 1 2 

and finally, from the estimated complex coefficients (with c q =-0.132): 

f M o 

+2 E |c T = 8.67 volts^ 

' 0 n=l n 
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The estimates give good results when compared to the true value thus in- 
dicating good quality and numerical stability. Refer to PROGRAM 05 for 
more details on this method of verifying results. 

B. PROGRAM 02 - Estimated Kronecker Delta Function 

1. I ntroduction 

This program evaluates the approximation to the Kronecker delta func- 
tion. It is included to show the high degree of accuracy in approximating 
Equation I2D.II. Figure 11-1 shows another application of the program. The 
practical usefulness of the routine is limited to analysis and testing of 
signal processing schemes. 

2. I nput/output data defi niti ons 

DELTQR = Matrix elements of the estimated Kronecker delta. 

M = Maximum row order. 

N = Maximum column order. 

Q= Row index or DELTQR. 

R = Column index of DELTQR. 

RF = Fourier period in seconds. 
tn = Observation period in seconds. 

3. Example 

The example run illustrates the result when the Fourier period and the 
observation period are equal. DELTQR is defined for both positive and 
negative indices. Diagonal elements are unity and off-diagonal elements 
are better than thirteen orders of magnitude smaller. 
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C. PROGRAM 03 - Estimated Spectral Amplitudes using Kronecker Delta 

1. I ntroduction 

This program is a simple example of the application of the estimated 
Kronecker delta function to computing spectral estimates. The algorithm is 
specialized to real (zero phase angle) amplitudes and must be rew ritten- for 
complex amplitudes if the user so desires. The routine is intended as an 
example of spectral mixing errors caused by an observation period unequal 
to a multiple of the Fourier period. Equation 11.20 is a prototype for the 
algorithm. 

2. I nput/output data definitions 

c (Q ) = Estimated spectral amplitudes. 

CTRDE(I) =1 nput { r true ') spectral amplitudes. 

M = Maximum index on input spectral amplitudes. 

N = Index corresponding to 2N + 1 samples. 

0= Index for double-sided spectral amplitudes. 

TN = Observation period in seconds. 

TF = Fourier! period in seconds. 

3. Example 

The data function has a Fourier frequency of 1 Hertz (Tp =1) and has 
been sampled 21 times in a period of 3.1 seconds. The true spectral com- 
ponents are 2.0 and 0.38 at I and 3 Hertz respectively. The estimated 
spectral components at zero frequency and intervals of (3.1) ” 1 Hertz are 
I i sted i n the pri ntout. 

At 0.968 and 2.90 Hz., the estimates are 2,038 and 0.356 respectively. 
Although these are reasonably accurate, spectral mixing has introduced 
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several large components at other frequencies. The component at 0.645 Hz. 
is -0.253 which is approaching the size of the true component at 3 Hz. This 
effect is present in every spectral estimator and must.be considered when 
evaluating the quality of the estimate. Notice also that the average value 
of the estimate is not zero. 

D. PROGRAM 04- D iscrete F ourier Transform A nalysis 

1. I ntroduction 

This program evaluates the discrete Fourier transform in its most fun- 
damental form. The complex coefficients evaluated are those necessary for 
the complex series representation (Equations A2.8 and A2.9). The algorithms 
evaluate Equations I2D.5 and 12D.6 and also do linear regression on the in- 
put data to remove an undesired linear trend. For purposes of better evalu- 
ation, the estimated complex amplitudes are used to generate an estimated 
time function with four times more time resolution than the input. 

SUBROUTINE LtfSPTM computes the complex Fourier coefficients based on 
the input time series and observation period. The algorithm is an imple- 
mentation of Equation 120.5. 

SUBROUTINE FSRS is used to generate a time series from Fourier coeffi- 
cients and time period information. The algorithm is an implementation of 
Equation 120.6. 

2. input/ output data definitions 

A ( I ) = In-phase or "real" component of the DIT output. 

B ( I ) = Quadrature or "imaginary" component of the DIT output. 

BZER0 = Intercept point from linear regression. 
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B I = 

C D B = 
CMAGDB = 


CM AG = 
CPHASE = 
CMAGN(I) = 
CPHASN(I) = 
DCV = 

FREQ = 
FSAVE(I) = 

FVALUE(I) = 
FX (I ) = 
LINREG = 


NLS = 

NS = 
SCALE = 

TDEL = 
TF = 

TN = 


Slope from linear regression. 

Component power in dB relative to unity total power. 
Magnitude of each spectral component in dB relative to 
the computed amplitude after scaling. 

Magnitude of the estimated Fourier coefficient. 

Phase of the estimated Fourier component in degrees. 
Normalized magnitude relative to unity total power. 
Phase i n degrees. 

Average or dc-value of the input time series. 

Frequency corresponding to each amplitude estimate. 
Initial input time series data saved for comparison 
purposes. 

Input time series data after linear regression. 
Estimated time series with, increased resolution. 

A user selected data preprocessing option. LINREG = 0 
chooses the input data function for spectral estimation 
while LINREG =1 selects the time series after linear 
regression modification. 

Number of data samples for use in SUBROUTINE LNSPTM. 
Number of data samples in the input time series. 

User selected scale factor for computing spectral ampli- 
tudes in relative dB. 

Data sampling interval is seconds. 

Fourier period in seconds. 

Observation period in seconds. 
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YMR = Average value of the input time series. 

Y F (I ) = Sample values of the straight line determined by linear 

regression. 

3. Example 

Time series data for the example waS provided by Mr. Wai Ling Tsui 
(Tsui, 1976). The time series is 41 samples of the output of a phase- 
switched interferometer. The radio source is Hercules A (3C348) and the 
data was taken 29 July 1975. 

The scientific objective is to obtain an estimate of the natural fringe 
frequency of the source. This is obtained by computing the digital Fourier 
transfonn of the time series data shown in Figure 170-1. The resulting 
estimated spectrum is given in Figure 17D-2. The plot is smoother than the 
printed data because zero-filling (concatenation) was used to improve the 
output resolution. The peak used to estimate the fringe frequency occurs 
at 2.11x10-^ Hertz. The true fringe frequency is predicted to be 1.95 x 
10-3 Hertz. This result is typical of what might be expected for dft spec- 


tral estimation. 




Figure 170-1. Time Series Data for the OFT Example 
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Figure I7D-2. DFT Spectral Estimation 
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E. PROGRAM 05 - Fast F ourier Transform Spectral Analysis 
1. I ntroduction 

This program implements a Fast F ourier Transform (FFT) routine to do 
spectral analysis. It is intended to be used for routine data processing 
and has features which include such preprocessing options as average value 
removal and data conversion to standard variable form. Output data options 
are also included that provide suppression of unwanted output during routine 
operations. 

The program will accept an arbitrary number of input data points and a 
resolution option provides an increase in output resolution (but not accur- 
acy) that is helpful in analyzing and plotting results. These features are 
accomplished by adding zeros (zero filling or concatenation) to the time 
series to bring the total number of input data values to a factor of.2 n 
where n is an integer. The FFT subroutine must have 2 n data points to oper- 
ate properly. Another feature is that, in addition to giving the usual 
aliased FFT output for 2 n points, the program does proper scaling and com- 
putes the true amplitude spectral density function. It also monitors the 
Nyquist frequency to guarantee that no output above the folding frequency 
is given. Finally, an estimate of the double-sided power spectral density 
function is given using the squared-amplitude definition. 

SUBROUTINE FFAST is adapted from a FFT routine given by Stearns (1975, 
Appendix B). The method is described as time decomposition with input bit 
reversal . 

A flow diagram describing the essential features of the program is given 


in Figures 1 7 E - 1 a and 1 7 E - 1 b . 
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XCI) • XSV(I) XCI) • XMDC(I) XCI) • XCI) 


FILL FFT ASSAY 
WITH DATA 



Figure 1 7 E - 1 a. PROGRAM 05: Fast Fourier 

Transform Amplitude Spectral 
D ensity 
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SUBROUTINE TTASJ 



Figure 17 E - 1 b. program 05 
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2. I nput data definitions 

NDC= The processing option for the input data function. The user 
has a choice of letting the FFT operate directly on the in- 
put data function X(T), (N DC = 0 ) , the zero-mean data function 
XMDC(T), (NDC = 1), or the data function converted to standard 
variable form XSV(T), (NDC =2). 

NRESL The resolution option for the FFT display. A nonzero sel- 
ection will add a power-of-two resolution increase for the 
FFT output. For example, nresl = 3 will increase the output 
display resolution by a factor of 2 3 . 

ns = The total number of time samples in the input data function. 
The user must dimension program variables accordingly. 

PTOPT = A printing option used to control the printed output of the 
program and used to suppress unwanted output. PTOPT is a 
binary word of the form: (NPTI, NPT2, NPT3, NPT4). NPTI 

controls the printing of the input data functions. NPT2 con- 
trols the printing of the FFT complex output. NPT3 controls 
the printing of the amplitude spectral density. NPT4 con- 
trols the printing of the power spectral density. Use a I 
if printing is desired and a 0 to suppress printing. 

SNAME = A name or label in FORMAT (5A4) used to identify the data 
record. 

XTDELT = Sampling interval for the input time series, seconds. 

X (T) = Input time series data. Units of volts will be assumed for 


simplified presentation. 
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3. Output stati sti cs 

DCV = Average value or estimated mean of the data function, Volts. 

7 

XBAR2 = Mean-square value of the input data function, Volts 

XVAR = Unbiased estimate of the variance of the input data function, 

V olts 2 • 

STDEV = Standard deviation. Volts. 

4. 0 utput data defi nitions 

A (N ) = In-phase or "real" component of the F FT output. Volts. 

B(N) = Quadrature or "imaginary" component of the FFT output. Volts. 

CMAG(I) = Absolute magnitude of the discrete spectral component ob- 
tained from, the discrete Fourier transform (DFT), Volts. 

FMAG(N) = Absolute magnitude of the FFT output, Volts. 

F-PHASE(N)=Phase angle of the fit output mod tt. Degrees. 

F R (I ) = Fourier frequencies of the spectral estimates, Hertz. 

KMAX = Total number of independent and ncmredundant spectral com- 
ponents. 

N = 2 * KMAX, the total number of data pairs used in the FFT sub- 

routi ne. 

S(l) = The estimate of the double-sided power spectral density func- 

2 

tion, Volts /HZ (Watts/as in 1 ohm). The printout gives only 
the positive frequency values. 

X X (I ) = The estimate of the double-sided amplitude spectral density 
function, Volts/Hz. The printout gives only positive fre.. 
quency values. 
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XMDC(I) =The input time series with the average (or 'de') value re- 
moved. 

XSV(T) = The input time series converted to standard variables which 
are derived from X (T) by subtracting the average value and 
dividing by the standard deviation. 

XTIM E(l) = Sample times for the input data function in seconds. This 
indexed time is used for plotting routines. 


5. Example 

An excellent example for illustrating many of the features of the FFT 
is given by Stearns (1975). The following example incorporates the same 
essential features. The data function may be written in the continuous 
form as: 

e _t si n t t ^ 0 

f(t) = (I7E.I) 

{ o otherwise. 

The Fourier transform of this function is determined from integral tables 
to be: 

J C ° • 1 

e-tsin t e-J<* dt = _ (17E.2) 

0 (2 - w ) + j2w 

The absolute magnitude of the Fourier transform is: 

lF(jW) I = (17E.3) 

(4+oj V 

This is the amplitude spectral density estimated by the program. The pro- 
gram prints only the positive frequency values because the transform is 
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symmetrical. For the reader's interest, the phase function is: 

e_( W) =ARCTAN r " 2w 2 J (17E.4) 

(2 - uu ) 

The program does not compute this result because we are assuming interest 
only in the spectral densities. 

The program input is 15 samples of Equation I7E.I starting at zero and 
going in steps of &t = 0.3 seconds. The estimated amplitude spectral density 
is shown in Figure I7E-2 and compared with the exact results given by Equa- 
tion I7E.3. The folding frequency for the given sampling rate is: 

ff =oL =1.67 Hz. 

As would be expected, the estimated amplitude spectral density shows the 
effects of aliasing due to components in IF{jW) I beyond the folding frequency. 
Also, the truncation effect of finite sampling introduces further high fre- 
quency components. The estimated amplitude for f =1.46 Hz is 106% too high 
while that for f =0.21 Hz is only 0.3% too high. These effects are common 
in most FFT spectral estimates. 

Figure I7E-3 shows the estimate when the resolution is increased by a 
2 

factor of 2 (NRESL = 2) . It is important to notice that the accuracy of 
the estimate remains unchanged and only the resolution is increased. Figure 
17E-4 shows the estimated power spectral density for the increased resolu- 
tion. 
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Figure I7E-2. Basic Resolution of FFT Estimate 
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Figure I7E-3. Increased Resolution for FFT Estimate 
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Figure I7E-4. FFT Estimate Proportional to 
the Power Spectrum 

Parseval's theorem provides a useful check of the stability and 
accuracy of the estimate and can be used to verify the correctness of the 
program. Using integral tables it is possible to show that: 

<S> -fill 

f (e _t si n t) 2 dt = in F = V 8 volts 2 7 Hz 

0 -■ 4+ w 

The integral in the frequency domain can be numerically integrated by the 
following routine to give 

&F X (0) + 2&F i X 2 (aAF) = 0.1249 vol tS 2 lHz 
n=l 
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where: 


^ 2KMAX At 


0.208 Hz. 


In the time domain, the result is: 


15 2 2 

At E x. =0.1248 volts 1Hz 
i=l 1 


These results may be compared directly with the standard deviation of the 
input data function. These agreements are very good even though aliasing 
has produced an estimate with considerable error near the folding frequency. 

Printed along with the amplitude spectral density function are the 
absolute magnitudes of the Fourier series components that would be used to 
represent the input data function. These amplitudes are related to the 
density furiction by the equation: 

c n = AFX (nAF) 

Refer also to Equation 11.13. 

An example of the effects of white Gaussian noise on the estimate is 
given in Chapter XIX and a detailed discussion presented in Chapter XII, 
Section D. 

F. PROGRAM 06 - Estimation of Single-Sinewave Parameters 
1. I ntroduction 

This program implements a digital Fourier transform amplitude spectral 
estimate for s. single frequency component with a frequency that may be con- 
tinuously varied. The input is assumed to be a single-frequency sinewave 
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plus bandlimited Gaussian noise. For a selected frequency below the Nyquist 
frequency, it will compute a "best" fit. The fit is "best" in the sense 
that the amplitude and phase represent an orthogonal component of a discrete 
Fourier series which can represent the input data function. . 

t 

The estimate is improved if the input data is first preprocessed to 
remove any linear trends. To guarantee a zero-mean data function, the pro- 
gram computes the mean value and removes it before doing the spectral esti- 
mate. The actual algorithms are based on Equations 13.13 through 13.18. 
These results will be presented here in a slightly different manner to pro- 
vide a better understanding of the estimator. 

A continuous complex amplitude estimate (not an amplitude density) is 
defined as 


fc ( f 0 ) 


1 +N 
= 2N±1 s x(kAt)e 
k— N 


-)■ kAt 
To 


(17F.I) 


where At =T^/ (2N + 1 ) • Discrete coefficients are defined at the frequencies 
corresponding to f Q =n/T . In general, the continuous complex amplitude is 
defined for any frequency between zero and the Nyquist frequency. 

If 6(f ) is represented in complex notation 

^( f i=s a ^y> + ^ ) (17F ' 2) 

the complex components can be computed from the following algorithms: 

1 2N + 1 2 

Qf(f ) =2N±1 2 x(kAt) cos fan (k - N - 1) At] (I7F.3) 

0 k=l o 

-l 2N+I 

8(f n ) = 2N±1 s x(kAt) sin t-(k- N - l)At] 

0 ' k=l 


(17F.4) 
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This method of defining the summation index forces the phase angle to be 
referenced to the center of the input time series. From Fourier analysis 
(Appendix II), the complex amplitudes are related to the usual a and b co- 
efficients as follows (Equation A 2.13): 


a 


n 



+C 


* 

n 


b n = j ( V C n : 


For the definition of continuous estimates this becomes: 


(17F.S) 

(I7F.6) 


a(f)=2Q'(f) (17F.7) 

b(f 0 ) =-2 3(f 0 ) (17F.8) 


The absolute magnitude of a complex amplitude estimate is: 

|6(f o )| = [c* 2 (f o ) + e 2 (f o )] % = \ Ca 2 (f o )+b 2 (f o )] % (17F.9) 

The absolute magnitude of a real frequency component is the sum of both 
positive and negative frequency contributions: 

d(f 0 ) = 2|c(f Q ) I (17F.10) 

The phase angle for the complex component is 

P(f ) 

0 c (f o > = ARCTAN (17F.II) 


while for the real component it is: 
*d< f o> = ^c< f o) - 270 ° 


(17F.12) 
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The terminology "real" and "complex" refer to the observable sinusoid and 
its double-sided mathematical representation. Finally the "residue" time 
series is defined as: 

x r (kAt) = x(kAt) - d(f ) sin [F (k- N - l)AtJ (17F.13) 

o 

The program output prints the input parameters and the input data func- 
tion statistics. The input time series and the zero-mean time series are 
also printed. Next, the magnitude, frequency, and phase of the best-fit 
sinewave are given. The "residue" time series is computed and its statis- 
tics are printed. Finally, the zero-mean input, sampled sinewave, and 
residue are printed. 

It should be emphasized that this technique is not superior to doing a 
complete FFT on the input data. It has advantages in its simplicity and 
selectable frequency but the FFT gives an estimate of all components as 
compared to only one. 

2. input/cxutput data definitions 

CMAG= Absolute magnitude of the complex coefficient defined in 
Equation 11.15 and evaluated at the chosen frequency. 

CPHASE = Phase angle of the complex coefficient in degrees. 

DMAG = Amplitude of the "best-fit 1 sinewave. 

DPHASE = Phase angle of the "best-fit" sinewave in degrees referenced 
to the center of the input time series. 

FREQ = Frequency in Hertz at which the "best-fit" sinewave is 
evaluated. 

F S ( I ) = Sinewave estimate evaluated at the sampling times. 
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NS = The total number of time samples in the input data function. 

SNAME = A name or label in FORMAT (SA4) used to identify the data 
record. 

TDELT Sampling interval for the input time series, seconds. 

TZERO = Period for which the sinewave approximation is evaluated. 

X ( I ) = Input time series data. 

XMDC(I) = The input time series with the average value removed to give 
a zero-mean time series. 

X R (I ) = This is a time series generated by subtracting the sinewave 
from the zero-mean input data. It is referred to as the 
"Residue" time series. 


3. I nput/output stati sti cs 

DCV = Average value or estimated mean of the time series. 

STDEV = Standard deviation. 

XBAR2 = M ean-square val ue. 

XVAR = Unbiased estimate of the variance. 

4. Example 

The time series data for this example is the same as used for PROGRAM 
04 except a linear regression analysis has been performed to remove any 
linear trend. The first printout, the input parameters, shows the total 
number of samples (NS =41), the period of the desired spectral component 
(T Q =49.26 seconds), and the sampling interval (At = 60 seconds). The sta- 
tistics for the input data function are computed and printed to be used for 
comparison with final results. For this example, the input time series 
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and zero-mean time series are equal because the data was previously processed 
to remove the mean value. 

The next output is the desired "best-fit" sinewave describing the in- 
put time series 

f =128.46 sin(|£ t - 1.839) (17F.14) 

NT 

where: 

T =4.926 x 10^ seconds, 
o 

Frequency = ! =2.03x10’^ Hz. 
o 

Phase = -105.34 degrees 

A statistical measure of the quality of this estimate can be obtained 
by applying Rayleigh statistics as discussed in Chapter XIX, Section C. 

First we assume that the time series is bandlimited Gaussian noise and test 
the hypothesis that the output amplitude spectral density was generated 
exclusively by noise. The expected value of the Rayleigh output is com- 
puted as: 

E$(f) } (NS x 2 ) % (17F.15) 

2 4 

For At = 60, NS =41, and x =1.4904 x 10 , the expected value becomes: 

E{x(f) } = 4.1566 X 10 4 = (f) % a x ' (17F.16) 

The estimated sinewave amplitude is converted to an equivalent amplitude 
spectral density by: 

X d -NSAt(^) (17F.17) 
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For DMAG= 128.5, the estimated amplitude density becomes: 

X. =*■ 15.8055 xlO 4 

d 

Since we know that a = 3.3165 x 10 4 , the estimated amplitude is at the 4.766 

A 

sigma-level on the Rayleigh distribution. The probability that an a priori 
selected sample at this frequency could be due to random noise is: 

Pf = P (X >4. 766 a ) =Exp[- \ (4.766) 2 1 = 1.17 XlO' 5 (17F.18) 

This is an extremely low probability. The probability that any amplitude 
estimated by a Fourier transform would exceed this value with only noise 
input is given by: 

P f =l- (l-P f ) NS = 4.789 xlO -4 (17F.19) 

This is also a low probability. A more detailed discussion of how to in- 
terpret estimated spectra is given in Chapter XIX, Section C. 

To give the reader a relative idea of the quality of the sinewave fit, 
the zero-mean data function, "best-fit" sinewave, and "residue" time series 
are plotted together in Figure 1 7 F - 1 . 
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Figure 17F-I. Single-sinewave Parameter Estimation 
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G. PROGRAM 07 - Maximum Entropy Spectral Analysis 
1. I ntroduction 

This program computes the maximum entropy estimate of the power spec- 
tral density function of a finite time series. A finite autoregressive 
series is used to model the input data function and the power spectrum is 
computed using the autoregressive spectral estimator given by Equation 
I5F.I. The estimates of the autoregressive coefficients are obtained from 
the Burg MEM recursion algorithm described in Chapter XV, Section G. 

The program is intended to be used for routine data processing and has 
been user oriented as much as possible. Processing features include user 
selected spectral output display resolution and autoregressive order. A 
selectable printing option has been included so that routine or unwanted 
output can be suppressed. Selectable data preprocessing options include 
removing the average value, linear trend removal, and converting the input 
to standard variables. 

Printed output includes the input data parameters, input time series, 
data function statistics before and after linear regression, time series 
after linear trend removal, standard variables, MEM starting values, inter- 
mediate coefficients (optional), summary of iterative results, maximum 
filter length, error power, estimated autoregressive coefficients, peak 
value of the estimated spectrum, total spectral power, and the normalized 
power spectral density estimate. 

The folding frequency is computed from the input data and the program 
inhibits the calculation or printing of any aliased output. The spectrum 
is scaled such that the output is the positive frequency half of a double- 
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sided spectrum. Since the spectral power has been normalized to unity, 
the program integrates the spectral density to obtain the total power in 
the estimated spectrum which may then be compared with unity. This provides 
a reasonable check on the quality of the estimate and the numerical stabil- 
ity of the algorithms. 

SUBROUTINE ATOREG recursively computes estimates of the autoregressive 
coefficients using the Berg algorithm. A detailed description of the rou- 
tine is given in the flow diagram of Figure I7G-I. The equation numbers 
refer to those equations in the text which are implemented in the sub- 
routine. 

SUBROUTINE MEMSPM computes the power spectral density estimate using 
the autoregressive coefficients from ATOREG. This algorithm implements 
Equation I5F .1 . 

A simple flow diagram describing the essential features of the entire 
program is given in Figures l7G-2a and 1 7G -2 b. 
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SUBROUTINE ATOREG 


T = 1 ,N 

P =P +X (T)**2 


PZERO= PIN 



I 


i 

i 

3 


The mean-square value is computed 
in the main program. (Equation 
I5G.5) 


Compute starting values for Bl and 
B2. (Equation I5G.I9) 


Compute the K = 1 values of AK(K) 
and P(K ). (Equation I5G.I2) 


(Equation I5G.4) 


Iterate 

Increment the order K. 

Compute the autoregressive coeffi- 
cients A K (T) using recursive equa- 
tions for increasing values of K up 
to K=MM. (Equations I5G.IO, 

I5G.II, I5G.15, I5G.16) 


Compute the highest order coeffi- 
cient A K (K ) for a Kth order process. 
(Equation I5G.12) 


Compute the mean-square prediction 
error. (Equation I5G.4) 


Figure I7G-1. Burg mem Recursion Algorithm for Computing 
Estimates of the A utoregressive Coefficients 
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SUBROUTINE ATOREG (CONT.) 



1 


| RETURN | 


Compute the estimated autoregres- 
sive coefficients for a Kth order 
model of the process. Printing of 
intermediate values is optional. 
(Equation I5G.3) 

Iterate 


Print a summary of the iterative 
results for increasing values of K. 


Print the final order and corres- 
ponding error power. Print all of 
the autoregressive coefficients 
for a Kth order estimate. 


Figure 17G-I. (Continued) 
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Figure 1 7 G - 2 a . Main Program Flow Diagram. 
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Figure 1 7G - 2 b. Subroutine Flow Diagr ams 
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2. I nput data 
FDELT = 

LINREG = 
M M = 

NS 

PTOPT = 


SNAME = 

XTDELT = 
X (I) = 


definitions 

Output frequency display resolution of the MEM spectral 
estimate in Hz. If the user selects FDELT =0., the program, 
will default to a value equal to four times the Fourier 
resolution. 

A user selected data preprocessing option. Spectral esti- 
mation may be performed using XSV(I), XNORM(I), XMDC(I) or 
X (I ) by choosing LIN REG = 3, 2, 1,0 respectively. 

User selected value for the maximum order of the hem model. 
This is the maximum number of autoregressive coefficients 
used to characterize the input time series. 

Total number of input data samples. User must dimension 
program variables accordingly. 

A printing option used to control the printed output of the 
program and used to suppress unwanted output. PTOPT is a 
binary word of the form: (NPTI, NPT2, NPT3, NPT4). NPTI 

controls the printing of the input data function and standard 
variables. NPT2 controls the printing of the summary of 
iterative MEM results. NPT3 controls the printing of the 
autoregressive estimation summary. NPT4 controls the print- 
ing of the MEM normalized spectral density. 

A name or label in FORMAT (5A4) used to identify the data 
record. 

Sampling interval for the input time series, seconds. 

Input time series data. Units of volts will be assumed for 
simplified presentation. 
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3. I nput data function stati sti cs 

x =XDC =Average value or estimated mean value of the data function, 
volts. 

2 2 
x = PT =Mean-square value of the input data function, vOlts • 

2 

ct x = PZER03 = Unbiased estimate of the variance of the input data 

2 

function, vOlts • 

a = STDEV3 = Standard devi ati on, volts. 

X 

4. L i near regression 

b Q = BZERO = Intercept point from linear regression. 

b. = BXI = Slope from linear regression 
2 

ct PZER02 = Variance of the data function after removal of the linear 

X 

trend. 

c = STDEV2 =Standard deviation. 

X 

5. 0 utput data def i niti ons 

A 1(1) = Starting value of the autoregressive coefficients for the 
MEM recursion algorithm. 

AK(K) = K-th autoregressive coefficient for an MM-th order process. 
FDELT = Output resolution of the MEM spectral estimate, Hz. 

F R (I ) = Frequency points at which the spectral estimate is computed, 
Hz. 

K = Index for the autoregressive coefficients. 

l = Index for spectral components. 

= Maximum spectral index number computed on the basis of 
al i asi ng. 


NMAX 
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P(O) = Normalized total power for the time series (equal to the 
variance). 

P(l) = Starting value of the mean-square estimation error for the 
MEM recursion algorithm. 

P(K) = Mean-square estimation error for the K-th order autoregres- 
sive estimate. 

PT = Total power (or variance) for the data function after linear 
regression. 

S(l) = MEM estimate of the output power spectral density (or second- 
moment density) function, watts/Hz. The printout gives the 
positive frequency half of the double-sided density function. 

SBIG = Largest spectral component in the estimated spectrum. 

SPLOT(I) =The relative power spectral density with respect to the 
largest component. 

XMDC(I) = Input data function with the average value removed. 

XNORM(I) =Data function after linear regression. 

XSV(I) = The input data function converted to standard variables after 
linear trend removal. 

XTIME = Sample times for the input data function, seconds. This 
indexed time is used for plotting routines. 

6. Example 

The time series data in PROGRAM 04 will also be used for this example 

because it provides a demonstration of the resolution differences between 

the DIT and MEM. A 9th order autoregressive estimate was chosen for the 41- 

.5 

point time series and the selected output resolution was 2.5x10 Hz. This 
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is approximately four times more resolution than the default value in the 
program. All the printing options were chosen so that the output features 
could be demonstrated. 

Since the raw data exhibits a pronounced average value as well as a 
linear trend, it was decided that the estimated spectrum would be computed 
onXSV(l), the input data function after removing the linear trend and con- 
verting to standard variables. The input data before and after removing 
the linear trend is shown in Figure I7G-3. The time series after pre- 
processing is shown in Figure I7G-4. This is the series for which the spec- 
trum will be estimated. Notice from the data function statistics after 
linear regression that the variance has been reduced to approximately 61% 
of its previous value. 

The MEM starting values are printed so the user may see the total 
power (mean-square value) in the data function and the error power in a 
first order estimate. A summary of iterative results is printed to illus- 
trate the trends in the reduction of the total error power and the extra- 
polated coefficient, [A K (K ): K=l, 2, .••MM}. The mean-square estimation 
error follows the typical trend of decreasing rapidly with order until some 
minimum level is reached. Figure I7G-5 illustrates this effect. 

The autoregressive estimation summary gives the mean-square estimation 
error for the chosen order (M M = 9) of the estimate and the best-fit auto-- 
regressive coefficients. The error power indicates a white noise contribu- 
tion of only 8.6% to the total spectral power. 
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The maximum entropy power spectral density estimation output prints 

the spectral density in watts/Hz as well as relative amplitudes in dB. The 

numerical integration to obtain total power produces a result very close to 

unity and this indicates good numerical stability. The largest spectral 

.3 

component occurs at a frequency of 1.925x10 Hz. Plots of the power spec- 
trum in watts/Hz and in relative dB are shown in Figures 17G-6 and 17G-7. 

These results should be contrasted with the DFT estimate given by 
PROGRAM 04. Comparing Figures 17D-2 and 17G-6 we see the spectral peak is 
much sharper for MEM. Also the fringe frequency estimate given by MEM is 
much closer to the true frequency (1.95x10 Hz.) than that given by the 
DFT. A further discussion of MEM resolution and this example is given in 
Chapter XIX. 
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Figure 17G-4. Time Series Standard Variables 


MEAN .SQUARE ESTIMATION ERR°R 
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Figure 17G-5. 


Estimation Error as a Function of 
the Order of the Estimate 
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Figure 17G-6. P war SpecCrum ; HER A 7|29|75 
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Figure 17G-7. P°wer Spectrum. HER A 7/29/75 
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XVIII. MAXIMUM ENTROPY AND FOURIER TRANSFORMS 
A. Introduction 

This chapter is devoted to a summary of the properties, similarities, 
and differences in DFT and MEM spectral estimation. These techniques are 
emphasized because they represent two of the most basic philosophies on spec- 
tral estimation: periodic extension in the time domain (DFT) and extrapola- 

tion in the autocorrelation domain (MEM). Throughout the presentation we 
will keep in mind the fact that spectral analysis is usually employed to 
detect periodicities in a data function and to determine the "strength" and 
"realness" of the estimated component. This presentation is by no means 
exhaustive in its coverage nor does it present the variety of viewpoints 
that are possible. Also, much of the material is redundant in the fact that 
it may be found in various forms elsewhere in the text. 


B. Fourier Transform Spectrum Summary 
Since the DFT is a discrete implementation of the Fourier transform it 
is helpful to discuss these properties first. A time function (x(t):-«^t£ 
-h*} and its amplitude spectral density function (x(juj) : ^ u)£-h»] form a 
Fourier transform pair (w=2nf): 


X(jw)=J* x(t)“j iut dt 


(18B.I) 


-j-CD • 

x(t) =J X(jw)e +jUJt df (18B.2) 

-X 

The Fourier transform does not exist for all possible types of data func- 
tions, a good example being white noise. If the Fourier transform is to 
exist, it must be piecewise continuous on every finite interval (Riemann 



311 


i ntegrabi I ity) and it must have a finite average value (necessary). A suf- 
ficient (but not necessary) condition is that it be absolutely integrable. 

Examples of functions which are most easily analyzed by Fourier trans- 
form techniques are single pulse waveforms and time functions modified by a 
finite data window. These types of transforms are commonly determined both 
analytically and numerically. 

The Fourier transform has the following useful properties: 

1. The real part of X(jw) is an even function in w; Re(X(jW)} 

R e(X ( - j w) }. 

2. The imaginary part of X (j w) is an odd function; lm(X(jw)} = 

-I m(X ( -j w ) }. 

3. The amplitude spectrum is an even function; IX (j w ) 1 = IX ( - j w ) I . 

Although not specifically a property, the majority of the functions analyzed 

using the Fourier transform are time limited: x(t) =0 for 1 1 1 ^ t max • This 

produces an amplitude spectrum of infinite extent. The absolute magnitude 
2 

squared, IX(jw) 1 , of the amplitude spectral density function is called the 
energy spectrum of x(t). This concept is used to relate time and frequency 
in the definition of Parseval's theorem: 


IX (jw) l 2 df = J I x ( t) l 2 d t 


(I8B.3) 


The energy spectrum of white noise does not exist because the Fourier trans- 
form i s undefined. 
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C. DIT Spectrum Summary 

A discrete Fourier transform pair can be obtained from Equations I8B.I 
and I8B.2 by using a Riemann sum approximation for the integrals. For a 
sampled time function, {x(kAt) : k = - N , -.n + 1, . . . , 0, . . • , n - 1, N}, containi ng 
(2N + l) samples of an infinite time series, the discrete Fourier transform 
(DFT) is defined by: 

+N 

X(jn&od = At E x(kit)e' jnAakit (18C.I) 

k— N 

x(kAt) = I® ^ X(jnAiu)e +:,nAuicAt (18C.2) 

There is absolutely no restrictions on the type of function used to obtain 
a DFT. However, if the DFT is used to make an estimate of the amplitude 
spectral density function cr energy spectrum of a continuous time function, 
x(t), the qual ity of the estimate will depend not only on the "true" proper- 
ties of x(t) but also on the limited ability of x(k&t) to faithfully repre- 
sent x(t). The commonly used esti mates for X(jw) and x(t) in terms of the 
sample function x(kAt) and its dft are: 


iN 


X(jui) = At £ x(kAt)e’^ uicAt 
k-N 

(18C.3) 

4N 

x(t) =|® 2 X(jnA(«)e +jnAuX: 

ZTT n-N 

(18C.4) 


The DFT has the following properties: 

1. The real part of X(jnAU)) is even. 

2. The imaginary part of x(jnAoi) is odd. 

3. The discrete amplitude spectrum, |x(jnAo>) I, is even. 
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4, The frequency interval of the estimate is: Au>= 2TT/(2N+ i)At. 

From the sample function x(kAt) , the estimated Fourier series is com- 
puted using (ISC. 4) and has the following properties: 

1. x(t) =x(kAt): t =0, +At, +2At, •••,+NAt. 

2. x(t) is periodic with period, TN = (2N + 1) At. 

The amplitude spectral density computed using (ISC. 3) has the following 
properties: 

1. It is periodic with period 2n/At, X (jw+ j2rr/At) =$ (jw). 

2. $(jw) =X*(-jw). 

3. The estimated amplitude spectrum is even, |x( juu) I = |x(-jcu) I , and 
is uniquely evaluated in the range, Osoj^rr/At. 

Parceval's theorem for the DFT is given by: 

+N ? l +N 2 

At S x (kAt) = / 9M + 1Uf . £ |x(jnAu>) I (ISC.S) 

k-N + n=-N 

An interesting property of the DFT concerns the concatenation of zero's 
to the data function. The new data function defined by 


y(kAt) =x(kAt) 
=0 


(k-0,+1,+2, *•* , +N } 
(k =±(N + 1), •••, jM] 


(ISC. 6) 


improves the resolution (not "accuracy") of x(jnAw) by decreasing aoj. This 
artifice produces a more continuous and hence pleasing display of the 
result. For a single time pulse there is no loss of accuracy but for a 
truly periodic function the period becomes distorted and an additional 
estimation error is introduced. 
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D. W iener-K hinchine Relations 

The concept of a oowsr spectral density function (as opposed to the 
amplitude spectrum) is based on the definition of a time autocorrelation 
function for a wide-sense stationary random process. The autocorrelation 
function and the power spectral density function form a Fourier transform 
pair called the W iener-K hinchine relations. The advantage of this concept 
is that the autocorrelation and power spectrum are defined for both periodic 
and random components. A disadvantage is that more numerical computation 
is required than for the DFT. The power spectrum of a random process is 
presented in Appendix I, Part F and in various other chapters. 

Applications of the W iener-K hinchine relations include the Blackman- 
Tukey method, the maximum entropy method, and the filtering method of spec- 
tral analysis. An important model for all of this methods is discussed in 
Chapter V. Most important are the time domain convolution i ntearal (Equation 
SA.I), the autocorrelation domain convol ution i ntearal (Equation SB. 5), and 
the I i near f i I ter power spectrum relationship (Equation SB. 8). 

E. MEM Spectral Theory Summary 

Any process with a bandlimited, finite, and continuous power spectrum 
can be represented by an infinite moving average (Equation 14.1). The fil- 
tering analogy of a roving average process is the linear transformation of 
white noise. The time series at the output of the filter can be represented 
arbitrarily close by a K-th order (K possibly infinite) autoregressive series 
(Equation 14.21). The maximum entropy method maximizes the entropy of the 
spectrum (Equation ISD.I) subject to the constraint that the inverse Fourier 
transform of the bandlimited power spectral density function (Equation ISF.l) 
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for a finite number of sampled autocorrelations be exact (Equation 15D.10). 
This is equivalent to a least-square error fitting of the time series with a 
finite number of autoregressive coefficients. Given enough data, this fit 
can be made arbitrarily close by increasing the order. 

F. MEM and DFT Comparisons 

The biggest contrast between DFT and MEM spectral estimators is that the 
former is an amplitude spectrum and the latter a power spectrum. They also 
differ in the fundamental way they approach the theoretical need for infinite 
data. The DFT assumes a periodic extension of the sampled data function, 
x(k&t) , while MEM estimates an extrapolated autocorrelation function based 
on an autoregressive model. Practical applications have shown that extra- 
polation results in an estimator with superior resolving power. Increases 
by a factor of 4-5 over DFT have been reported in the literature (Ulrych 
et. el . , 1973). 

For bandlimited random processes, the power spectrum estimate of MEM 
converges with increasing amounts of data while the DFT estimate does not. 

The spectral amplitudes for the DFT estimate, |x(jn&uj) I, are always Rayleigh 
distributed for Gaussian noise no matter what the record length. 

The DFT spectral estimate is periodic, bandlimited, and discrete be- 
cause of the periodic extension assumption and the Nyquist sampling theorem. 
The MEM spectral estimate is periodic, bandlimited, and continuous because 
of the autoregressive model. Both can be designed to give an estimate that 
is continuous in frequency although practical resolution is usually limited 
by the required computing effort. 
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The DFT estimate is "optimum" in the sense that x(t) (Equation I8C.4) 
exactly reproduces the original time series, x(k&t), at the points t = kAt. 
The mean-square error of the time domain representation is thus constrained 
to zero: 

E{[x(kAt) - x(kAt) J 2 } =0 (18F.1) 

Because the estimator was also constrained in frequency (bandlimited) it 
also provides the "smoothest" extrapolation between data points, x(kAt), 
consistent with the restriction that x(t) be periodic. The accuracy with 

A 

which the spectral estimate, X(jnAw), agrees with the "true" spectrum, 

X (jw) = < J{x(t)}, depends upon how well x(t) is modeled by a bandlimited 
periodic function. 

The MEM estimate is "optimum" in the sense that the spectral entropy 
(Equation ISD.I) is minimized while constraining the estimated autocorrela- 
tion function to be the exact inverse Fourier transform of the power spec- 
tral density. The mean-square error in the time domain is minimized by 
choosing autoregressive coefficients which minimize (Equation 13.36): 

ECCx(kAt) - x(kAt)] 2 } =c^ (18F.2) 

An interesting parallel between the DFT and MEM can be developed using 
ideas presented in Chapter XIII. Equation 13.9 represented the Fourier 
series in a form similar to autoregression. The autoregressive approach 
starts with the best linear one-step predictor in a least-squares sense: 

a K 

x(k+ 1) = £ a x(k+ 1 - n) 

n =0=1 1 


(18F.3) 
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Using a complex representation for a sinusoid of frequency, f, we have: 


x(k) =Ae 


+j2rrfkAt 


ISF.4) 


If, ideally, the prediction error is to be zero, x(k+l) - x(k+l) =0, then 
substituting Equation ISF.4 into 18F.3 gives the condition for the complex 
coefficients as: 

Ae +j2TTf(k+l)At = 2 a e +j2rrf (k+l-n) At n^p ^ 

n=l n 


This can also be written as: 

1. s 

n=l n 

The solution for a n from (I8F.S) is given as: 

a = I +j2nfnAt 
n K 6 


(ISF.6) 


(ISF.7) 


These are the complex coefficients needed to perfectly predict (or filter) 
the single-frequency sinusoid. 

For more than one sinusoid the solution is much more involved. Consider 


the representation 

M + jX k 

x(k) = S A e P (I8F.8) 

p=l P 

where \ =2rrf At. The condition for the complex coefficients now becomes: 
P P 

M +j\ (k+l) K M +j\ (k+l-n) 

Z k e p = £ a £ A e P 
p=l P n=l n p=l p 


The solution for a n is given by 


(ISF.9) 
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1 m +jX (k+1) M +jX (k+l-n) 
a = £ [ Z A e p J l E A e p J (ISF .10) 

n K P =i p p=l P 

which reduces to (ISF. 7) for M=l. Notice the k dependence of a n when more 
one frequency is involved. This means that a n depends on time which is not 
allowed for autoregressive coefficients. In conclusion, mem cannot repre- 
sent the Fourier series given by (18F.S). These representations may be con- 
trasted with the Fourier series representation of Equations 11.9 and 11.10 
to determine further comparisons. 
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XIX. INTERPRETATION OF ESTIMATED SPEcrRA 
A. Introduction 

A study of the properties of the various spectral estimators is only 
part of the task necessary for the practical application of spectral theory 
in research. Even after careful selection and implementation of a good es- 
timator one is faced with the task of interpretation of the computed results. 
In this chapter we will concentrate on the DFT (FFT) and MEM techniques. 

The important properties that affect the interpretation of the results of 
these estimates are presented along with practical examples. Two of the 
most important concepts presented are the detection criterion for a sine- 
wave in Gaussian noise using the FFT and the effects of data length and 
autoregressive order on the estimated frequency for MEM. The summaries on 
resolution limits and common estimation errors should be especially helpful 
for designing the data taking and processing methods. 

B. D ata F unction M odels 

It is helpful in the interpretation of estimated spectra to keep in 
mind a suitable model for the process. Three possible models were suggested 
in Chapter XIII that seem to be able to describe most empirical data func- 
tions. These models are: 

1. Fourier series plus bandlimited Gaussian noise. 

. . 2m . 

+M +J T 

y(t)=?(t)+E ce (19ES.I) 

n-M n 


2. Autoregressive series of order K. 
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K 

y(k) =C(k)+ Z a x(k - n) (19B.2) 

n=l n 

3. Convolution of a known function and bandlimited white Gaussian noise. 

y(k) =z(k) *r](k) (19B.3) 

Strictly speaking, the amplitude spectral density does not exist for 
these models because they de not have Fourier transforms. As a. practical 
matter however only a limited time record is ever observed and thus the 
actual empirical data model is the product of y(t) and a suitable time do- 
main window: 

x (t) = y(t) • w (t) 

w (t) ^ 0 It | d t m 
= 0 otherwise 

The Fourier transform of x(t) exists for all of the suggested models. The 
effect of the window function and random noise on ,the estimated amplitude 
spectrum are a topic of discussion in this chapter. 

The power spectral density function exists for all the models of y(t) 
because they all have well-defined autocorrelation functions. The "unwin- 
dowed" power spectrum is usually referred to as the "true" spectrum and 
the quality of the estimate is judged against the "true". This is not a 
really fair comparison in many cases because of the degrading effects of 
data windowing. Whenever possible, an estimate should be compared to the 
spectrum of the "windowed" function. 


(19B.4) 

(19B.5) 
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For purposes of reference, the power spectral density functions of 
the models will now be given. For all cases, the spectral models will be 
obtained from the following: 


E (y(t) Y(T-t)}=R y (T) 

(19B.6) 

°P(R y (T)}=S y (f) 

(19B.7) 

^(EClCt) § (T-t)}}«S § (f) 

(19B.8) 


The spectra of the models are: 

1. Fourier series plus band I i m i ted Gaussian noise. 


Tl v l n 

S (f) =S F (f) + E |c r 2 nS(g - f) (19B.9) 

Y * n=-M n T F 


2. Autoregressive series of order K. 
S (f) = 

y 


S § (f) = c At |f!^^ 

= 0 otherwise 

^ = 2At 



(19B.I0) 


(19B.II) 


3. Convolution of a known function and bandlimited white Gaussian 
noise. 


S y (f) =S z (f)S ? (f) 


(19B.12) 
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S z (f) = H (jw)H (jw) 

(convolution f i Iter) 

(19B.13) 

S § (f) =o| At 

Hi 

(A 

eT 

(19B.14) 

= 0 

otherwise 


£ N = 2At 




There are many pathological functions which do not fit these models 
and whose spectra are not well-estimated but for the majority of noisy 
data, these models are adequate. Also, there seems to be a reluctance on 
the part of some theorists to accept the noise model restrictions of finite 
bandwidth and Gaussian distribution on the noise model. Bandlimiting is 
not a problem either conceptually or practically as all measurement tech- 
niques cause bandlimiting. The Gaussian assumption is harder to justify 
conceptually but for practical applications the C entral Limit Theorem 
usually applies because of all the linear operations associated with signal 
processing. In most communication and signal processing systems the narrow- 
band filtering (linear filtering) generates Gaussian noise. 

C. DFT and Gaussian Noise 

The DFT is still commonly used for analyzing noisy or almost random 
data in spite of the fact that nearly every book on signal processing cau- 
tions that the DFT estimate for random noise does not converge. This mis- 
use is probably a result of the following circumstances: 

1. The FFT is readily available and the Fourier transform method of 
estimating an amplitude spectrum is widely understood. 
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2. The autocorrelation method of estimating a power spectrum is more 
difficult and the B lackman-Tukey approach is not as widely under- 
stood. 

3. The data window effect (Equation 19B.4) allows the existence of a 
dft for x(t). It also allows for a statistical interpretation of 
the estimace and for spectral smoothing techniques. 

The application of the dft to a limited time record of bandlimited Gaussian 
noise will now be presented. 

The complex Fourier amplitudes for a random signal defined by U(k): 
k =0, +L. ±2, ••• , are given by the complex series: 


e 

n 


1 -W 
2N + 1 k^N 


S(k)e 


2rr 

2N + 1 uk 


(19C.1) 


The sampling interval in time is and is represented at times, t =k£t, 
by the index k. An alternate representation for (19C.1) is given by 


A 

C 

n 


1 

2N + 1 




where 

a =|( 0 ) + S CC(k) +C(-k)] cos (2N ± T 1 nk) 

n k=l 

3 = E Cf(k) - f(-k) ] sin (2N.ll uk) 
n k=l 


(19C.2) 


(19C.3) 

(19C.4) 


A 

C 


n 


A 
— C 


-n 


(19C.S) 
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a r and | are both Gaussian because the operations described by (19C.3) and 
(19C.4) are just scaling and linear combinations of the Gaussian random 
variables) §(k). Even when ?(k) is not Gaussian the central limit theorem 
applies and the linear combinations of many samples makes a n _and £ n tend 
to be Gaussian. 

The discrete amplitude spectrum is estimated by scaling the absolute 
value of the complex Fourier amplitudes (Af = 1/(2N + 1) At) : 

|x(nAf) I =(2N +l)it|c n I (19C.6) 

In terms of the a n and $ , the estimated amplitude spectrum is: 

|x(nAf) | = At(o- 2 + (19C.7) 

A 

From statistical communication theory (Whalen, 1971) we know that X(n£f) 
represents an envelope function and hence is Rayleigh distributed. Since 
the Rayleigh characteristic does not depend upon the number of samples used 
in the estimate... the amplitudes remain Rayleigh distributed no matter how 
much data is used. For this reason the amplitude spectral estimate is said 
to be nonconvergent. 

Figures 19C-I and 19C-2 show the estimated amplitude spectra for sample 
records of a bandlimited white Gaussian noise process. The "parent" Gaussian 
process has the following properties: 

1. Mean: E{§(k)}=0. 

2. Mean-Square: E{? 2 (k>}=l. 

3. Variance: E{(?(k) - E{?(k) })2}= 1. 

4. Power Spectrum: S^(f)=l/At, Ifl^^. 
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The first sample data record with M=32 points has the following properties: 

1. Mean: <|(k)> = 3.25 x 10- 2 

2. Mean-Square: <| 2 (k»=1.39 

3. Variance: <(§(k) - <|(k)>)2> = 1.43 

The nonnegative half of the estimated amplitude spectrum shown in Figure 
19C...1 has the following properties [n=0, 1, 2, .••, - I)}: 

1. Spectrum: Rayleigh distributed 

2. Mean: <IX (nAf) 1>= 5.94 

3. Mean-Square: <|x(n^f) 1 2 >= 43.8 

4. Variance: « |x(nAf) | - <|x(nAf) 1»2>= 9.06 

The second sample record with M= 256 points has the following properties: 

1. Mean: 4(k)>= -1.52 x 10-2 

2. Mean-Square: <? 2 (k» = 0.979 

3. Variance: <(?(k) - 4(k)>)2>= 0.982 

The nonnegative half of the estimated amplitude spectrum shown in Figure 
19C-2 has the following properties [n=0,l, 2, •••, (j - I)}: 

1. Spectrum: Rayleigh distributed 

2. Mean: <1 X (nAf) 1>= 13.8 

3. Mean-Square: <|x(nAf) |' 2 >= 250. 

4. Variance: « |x(nAf) 1- <|x(nif) 1»2>= 61.4 

Parceval's theorem for the DFT as given in (18C.5) can be rewritten in 
terms of the present quantities to be: 

AtM<§ 2 (k)> =“ Af[M<jx(nAf)| 2 >- |x(0) | 2 ] (19C.8) 

For At =1 and Af = 1/MAt the estimates above (for M = 256) give 
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nAf, Frequency 


Figure 19C-I. Estimated Amplitude Spectral 

Density for Gaussian Noise (FI =32) 
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and: 


At M<§ 2 (k)> 251. 


Af[M<|X(nAf) | 2 >- |X(0) | 2 ] - 250. 


This is in excellent agreement, we also know from theory that, for a truly 
Rayleigh distribution, the mean and mean-square values should be related as: 

E{|x(nAf)! 2 }= £ E 2 {|x(nAf)]} (19C.9) 

For estimates, the. approximation is: 

<|x(nAf) j 2 > =“£ <|x(nAf) !> 2 (19C.10) 


For the estimate shown in Figure 19C-2 the ratio of right to left-hand-side 
of (19C.I0) is 0.970 which is a three-percent difference. 

Combining Equations 19C.10 and 19C.8 gives an'important result that 
relates the average value of the output amplitude spectrum to the input 
data. The theoretical result is 

E{|x(nAf) [} = (f) % [(At) 2 HE{§ 2 (k)} 4- ^ ] % (19C.11) 


while for estimates the approximation is: 

<|x(nAf) l> - (f) % [(At) 2 M<| 2 (k) + 
{n=o, i, 2, .... (| - l)} 


(19C.12) 


The reader should again be reminded that all the equations are written in 
terms of only the nonnegative part of the double-sided amplitude spectrum. 



X|nAf), Amplitude Spectral Density 
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WlLITODE SPECTRAL DENSITY tUNCTION 
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Figure 19C-2. Estimated Amplitude Spectral 

Density for Gaussian Noise (M = 256)' 
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Using the notation 


% = \ r(At) 2 M<t 2 (k)> + 

(19C.13) 

it is possible to express the characteristics of the Rayleigh 

di stri buti on 

compactly as: 


1. Mean: <|x(n£f) l> =» (5)^ 

(19C14) 

2. Mean-Square: <|x(n^f) | 2 > =- 2a 2 

(19C.15) 

3. Variance: « Jx(nif) 1 -<|x(n£f) l» 2> =■ (2 - ^)a 2 

(19C.16) 


For the example with M= 256, Equation 19C.13 gives (a 2 = 125.3). 

In this section we have shown that the amplitude spectrum estimates 
are Rayleigh distributed both in an ensemble sense and across frequency. 

The statistical properties of a dft estimate of the amplitude spectrum 
of Gaussian noise can be used to develop a simple test for the detection 
of a periodic component in the spectrum both over frequency and in an 
ensemble sense. 

First, at an a priori selected frequency a£f, the amplitude estimates 
are Rayleigh distributed over an ensemble of spectral estimates. The 
probability that a single estimate, I, from one ensemble function will 

3 

exceed a selected threshold X Q is given by: 

p 0 = p( 1*X Q ) = expC-X o 2 /2aJ] (19C.17) 

This is analogous to the well-known "false alarm" probability for statisti- 
cal detection (Whalen, 1971, Ch. 8) when using a Rayleigh envelope function 
(Whalen, 1971, Ch. 4). The detection criterion for testing an ensemble of 
spectral estimates for a periodic component at a£f is developed by selecting 
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a confidence limit using (19C.17) and (19C.13) and testing all X 's against 

3 

the computed threshold X Q 

Second, for a single sample record of the estimated amplitude spectrum 
the amplitude estimates are Rayleigh distributed over frequency. The 
probability that any single estimate, |x(n&f)J, will not exceed a selected 
threshold X Q is given by: 

P(lx(nAf)Ux o )=(l-P 0 ) (19C.18) 

This is simply one minus the false alarm probability. Using the theorem 
of joint probabi I ity and the multiplication theorem (Beckmann, 1967, p. 19) 
leads to the probability that any. of the M/2 independent amplitude estimates 
will not exceed a threshold X Q : 

p(x r x 2 , ...,x M/2 ^x o ) = (l-P o ) M/2 (I9C.19) 

Likewise the probability that one or more amplitudes in a single spectral 
estimate will exceed X Q is given by: 

p =1 - ( I - P )M/2 (19C.20) 

o o 

Equation 19C.20 can be used to test the largest spectral component to 

A 

see how likely it would be to achieve that given level. If max(X(nAf)} ex- 
ceeds some chosen confidence threshold X Q , we can use this test to determine 
the likelihood of a periodic component. As an example, for the record 
M =256, the largest amplitude is: 

max(|x(nAf) 1} = IX (10 A f) I =37.4 


(19C.2I) 



331 


2 

Computing a Q using (19C.13), the test probability using (19C.17) is: 

P Q = ex p( - (37.4) 2 /2(125.3)] =3.77 x 10' 3 (19C.22) 

The probability that at least one of the amplitudes would be as large as 
X(lOAf) is: 

a 128 

P =1 - (1 - P 0 ) =0.38 (19C.23) 

This is a very likely occurrence so we can assume that the spectrum repre- 
sented by Figure 19C-2 is due exclusively to random noise. 

The threshold for a confidence limit of 0.01 is obtained by solving 
Equa t ions"19C . 20 and 19C.17 for X Q . The solution is as follows: 

f>=0 .O l=l-(l-P) 128 

P =7.85 XlO' 5 
o 

a 2 = 125.3 
o 

X =48.7 
o 

If any of the estimated amplitudes had exceeded this level it would have 
been strong evidence for a periodic component in the data. This technique 
is recommended as a method of detecting a periodic component in noisy data 
by using the DFT. 

A second example will now be given that includes both sinewave detec- 
tion and the effect of increasing the total number of phase coherent samples 
(as discussed in the next section). A test signal of the form 


y(k£t) = S(kAt) + A si n (2rTfk&t + 0.77) 


(I9C.24) 
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was generated using; Gaussian noise with zero mean and unity variance, 

A =0.707, and f=0.249. PROGRAM 05 was used to obtain amplitude spectrum 
estimates for data records of various lengths. The results and conclusions 
will now be discussed. 

Table 19C-I is a summary of results for various data samples. The 
Rayleigh mean and standard deviation are shown along with the confidence 
thresholds computed for the 0.05, 0.01, and 0.001 confidence limits. The 
largest amplitude, its frequency, and probability of occurrence are also 
shown. Figure 19C-3 shows plots of the amplitude spectral density function 
for the various phase-coherent sample lengths. The effect of the sinewave 
component "growing" out of the noise is very evident. 

The sinewave component is easily detected for M = 256 as shown in 
Figure 19C -3(d). The probability of occurrence of a component this large 

5 

from noise alone is only 1.5X10- • This corresponds to a DFT'spectral 
signal-to-noise ratio (Chapter XIX, Section E) of 31.9 or +15.0 dB (Equa- 
tion 19E.4).. These numbers agree quite well with envelope detection theory 
(Whalen, 1971, Chapter 8). From this we conclude that enough samples should 
be taken so that SNR (Equation 19E.4) is greater than 12- 15 dB. 

This last example was intended to illustrate the effect of increased 
sampling on the detection of a sinewave component in noise and to indicate 
the number of samples necessary for a good probability of detection. 



Table 1901. Summary of Results for the Example of the Detection of a Sinewave in Gaussian Noise. 
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@0.95 

@0.99 

32 

5.32 

4.24 

14.4 

16.3 

64 

7.62 
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10.9 
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Figure 19C-3. Example Results for the Detection of a Sinewave in Gaussian Noise 
as the Number of Coherent Samples is Increased 
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Figure 19C-3. (Continued) 
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D. DFT and Spectral Averaging 

In Chapter XII the sampling and estimation problem was discussed and 
some groundwork was done on spectral averaging for noisy periodic signals. 
In Section C of the present chapter, the DFT amplitude spectrum estimate 
was characterized for a bandlimited Gaussian noise sample. In this section 
we will look at simple methods of spectral averaging and how they affect 
the estimate of the amplitude spectral density function. 

. For a Gaussian noise input, the amplitude estimate given by Equation 
19C.7 has an average value given by Equation 19C.14. For large samples 
the mean converges to: 

E{|5(nAf)!3=At(fME[§ 2 (k)}) % (19D.I) 


The mean value depends on the sampling interval, the total number of samples, 
and the variance of the input noise. 

Now consider a finite data function {x(kAt) : k=0,±l, +2, ••• , } 

represented by Equation 19B.I and a rectangular data window function of 
width, T N =(2N + l)At: 


4M 




x(k£t) = §(kAt) + Z c e 
m— M m 


(19D.2) 


The amplitude spectral density is estimated using the sum of the DFT of 
|(kAt) and the spectral mixing formula (Equation 11.19): 


+N -j 2TT 

X(nAf) = At Z S(kAt)e 2N+I 
k-N 

-Hi 

+ ^ Z - nAf) 

" m=-M m T F 


nk 


(19D.3) 
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If we represent the second term of (19D.3) by the estimate, A(n&f), of the 
"true" spectrum and the first term by Equations 19C.3 and 19C.4 the result 
i s: 


X(nAf) =4t(0' n - j 3 n ) + A(n&f) (I9D.4) 

Since the estimate A is complex it can be represented as 
a "j e n 

A(nAf)=A n e (19D.5) 


and the estimated amplitude spectrum can be expressed in complex form as: 

X(nAf) = CACa n +A n cos - jCAtS n +A R sin (I9D.6) 

The terms A R and are really the noiseless estimates of |x(nAf) I and 
arg{x(n£f) } discussed "in Chapter XII. These estimates are corrupted only 
by the spectral mixing effect. The" terms and are Gaussian random 
variables which represent the effects of noise. To adequately represent 
the effects of record averaging, the terms in Equation 19D.6 need a sub- 
script denoting the record number. If the record number is denoted by the 
subscript m then the following double subscript notation is needed: 


* m (°4£) =Cit * om +A nm cos €j - Hit 8 nm +A om sina nm ] (19D.7) 

The vector or coherent average for R records is expressed as: 

a . . 1 R A IR i 

Ave[X (nAf) } =[&t -R £ 0 i + -R £ A cose | 

m m=l nm m=l nm tun 


■ R R 

- i Cit -R £ % + R £ A si n e ] 

m=l nm m=l nm om 


( 1 9 n . 8 ) 
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For large numbers of records the noise averages out and the result may be 
written in abbreviated notation as: 


A ve(\ (n6f)}=A ve(^ cos ^ }- j Ave(A n si n^} 


(19D.9) 


The algebraic or i ncoherent average is obtained by averaging the mag- 

* 

nitude and phase separately. The magnitude and phase of x(nAf) are: 

Lta2 


|x(nAf) l = [A 2 + (At) 2 (^ + 8 2 ) + 2At A^^ cos e R + $ n si n ej f 2 


a -(At 8 +A n si ne) 

Arg{x(nif) } =A rctan[ _Q Q Q-) 


(At a + A cos e 

n n n 


(19D.10) 

(19D.II) 


Since only the average magnitude is desired for an amplitude spectral den- 
sity estimate the two types of averages give the final results: 

1. Vector or coherent average. 

Ave( |x(ni£) | } = [Ave{A R cos ^ } 2 +A ve(A n si n e R } 2 ]^ (19D.12) 


2. Algebraic or incoherent average. 

Ave{ Jx(nAf) | } = [Ave(A 2 } + (At ) 2 Ave{or 2 + 8^ }]^ (19D.13) 

The quality of the estimate of the amplitude spectrum is affected both 

by the observation period (or the total number of samples for fixed At) 

and by the number of records averaged. Equations 19D.12 and 19D.13 are the 

final results assuming that enough records have been averaged that the 
variability of the estimated average can be neglected. 

According to the spectral mixing theory of Chapter xi, as the observa- 
tion period TN increases, the estimated converges to the "true" amplitude: 
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” i e n 

Ae n ^X(nAf)=TC (19D.14) 

n n 

and the coherent average gives: 

Ave{ IX (nAf)J } IX(nAf) I (19D.15) 

_rf the records are not phase connected, the phase angles en become random 
functions of the record number with a uniform distribution (-n,+rr) and the 
coherent average becomes zero. Normally the phase connection will be ap- 
proximately correct and 6 n will have a Gaussian distribution around the 
mean angle. If phase connecting cannot be guaranteed the coherent average 
should not be used. The coherent average is a fickle performer as an es- 
timator and must be used with caution. 

To assess the performance of the incoherent average we need to express 
it in an alternate form by using Equations 19C.7 and 19C.14 to give: 

A ve{o£+ £"} = — — (§)oJ: (190.16) 

n " at ) 2 2 ° 

This is used in 190.13 to obtain the alternate representation: 

Ave{|$(nAf) ! }= [Ave[A^}+ (190.17) 

2 2 2 

From (190.3) we see that A increases as T„ while from (19C.13) o increases 

n « o 

only as (M =TN ! At) • Since A n is always positive, the estimate for large 
TN converges to: 

Ave{ Jx(nAf) | } =“ Ave(A n } ( 1 9 D .18) 

For Ave{A n } =0, the estimate of (190.17) converges to that of (19C.14). 
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The amplitudes A nm are statistically variable because the samples are 
not phase connected and because of spectral mixing effects. Because the 
phase reference for several arbitrary records is random and approximately 
uniformly distributed, A nm will have a Gaussian distribution about some 
average value that has a fixed bias from the true value, |x(nAf)|. 


E. DFT Signal-To-N oise Ratio Concepts 
The concept of a spectral signal-to-noise ratio seems like a natural 
extension of the presentation in Section D and may contribute to a better 
understanding of the spectral averaging concept. If we define a signal-to 
noise ratio for a Fourier frequency n&f as 


SNR = 


2 | c i Z 
1 n 1 

5 <§ 2 (k)> 


(I9E.1) 


and use the relations, 


i ** 2 At t n <§2(k)> U9E.2) 

|cj -jr |x(n A f)| (19E.3) 

the SNR at n&f becomes (M= V * C) : 


SN R ^ 

% 

The average value of the estimated amplitude A n 
correction factor y times the "true" amplitude: 

} n r 


(19E.4) 

can be represented as a 


Ave{A n }=y n |x(nAf) I 


(19E.5) 
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If we identify the total number of samples as M=T N /At then Equation 
19D.17 may be written as: 

Ave { |x(nAf ) |} = (f) \ [l + (£) vjj SNR^ (19E.6) 

This equation expresses the average amplitude spectral density as a func- 
tion of the signal-to-noise power ratio for a spectral component repre- 
sented by the complex Fourier coefficient, c . 

For a sufficiently long sample of a model consisting of Gaussian noise 
with a single periodic component represented by c , the spectral mixing 
correction factor, y is unity for n = m and zero for all other n. This 
leads to the approximations: 

A ve( |x(n&f) | } - (j) % a o n^m (19E.7) 

Ave(|x(«nAf)|} - (f) % a o [l+(|) SNR m ] % (19E.8) 

For a single sinewave we will briefly introduce another concept with the 
help fo Equation 19E.8. Applying Parceval's theorem (Equation I8C.S) we 
get 

+N 2 1 +N I-. 2 

At £ x z (k) = (2N + 1) a#- s Ave{ X(nAf) 1} (19E.9) 

k-N ^ n-N J 

or after many manipulations: 

M(At)^ <x^(k)> = — a^Cl + ~ SNR ] (19E.IO) 

L O TT — — UI 

The presence of a sinewave increases the mean-square value of x(k) as in- 
dicated by Equation 19E.10. If the removal of a spectral component mAf . 
decreases <x (k» by an amount that is statistically more significant than 
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the removal of a random component, then (19E.IO) could be the basis of a 
criterion for detecting a single sinewave component in noisy data. 

The author plans to publish additional work with these topics in a 
future paper. 


F. MEM in the Z-Transform Domain 

In Chapter XV it was shown that the maximum entropy spectral estimator 
could be represented by (Equations 15D.38 and 15D.39): 


A 

S (f) 


X 


* tP k 

Q(Z)Q*(Z) 


AtP k 
Iq(z) I 2 


(19F.1) 


where: 

Z=e+j2TTfAT | f | ^ _J_ (19F.2) 

ZAt 

K -k 

Q(Z) = 1 - S V k > z (19F.3) 

k=l ^ 


Remember that at is the sampling interval in the autocorrelation domain, 
that a^(k) represents the coefficients for a K-th order autoregressive 
model of the time series, and that is the minimum mean-square error of 
the one-step prediction. The properties of the spectral estimator can be 
related to the properties of the complex polynomial Q(Z) and the associated 
coefficients, a^k). 

The orders of the polynomial and the process are always the same. Q(Z) 

also represents the z-transform of the "best" least-squares linear predic- 

2 

tion filter for the process. For this reason IQ (Z)l will be called the 
absolute value squared of the filter transfer function or simply the power 
transfer function. We will use the abbreviated notation 
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G(f) = lQ(Z) I 2 = |Q(e +j2TTfAT ) | 2 


(19F .4) 


to indicate that Z is evaluated on the unit circle so that G(f) is real. 
The notation is not precise but generally there is no confusion as to 
meaning. Another notation option will be the substitution of hT^i/ 2 t^ 
into some of the equations so that the effect of the Nyquist frequency 
limit can be displayed directly: 


Z =e 


4-jnf/^ 


l £ l ^ 


(19F.5) 


As a simple example we will look at a first order process with associ- 
ated polynomial and power transfer function: 

Q(Z) =1- a, ( I ) Z _ 1 (19F.6) 

\ 

|Q(Z) I 2 = [1 - a 1 (l)z' 1 ][l - ai (l)Z +1 ] 

= -a 1 z‘ 1 +Cl + a 2 (l)]-a 1 Z +1 (19F.7) 


2 

The zero of Q(Z) occurs at Zl =a 1(1) and those of IQCZ) 1 at Zl 2=a. (1), 

J ' 

I /3| (I). Figure 19F-I shows the estimated power spectral density for dif- 
ferent values of alCI). 

Now consider a second order process: 

Q(Z) =1- a 2 (l)Z _1 - a 2 (2)Z' 2 (19F.9) 

lQ(Z)i 2 =-a 2 (2)Z' 2 - a 2 (l) [ l- 3 2 (2)]Z' 1 

+ [l + a 2 (l) +a 2 (2) J - 3 2 (1)[1- 3 2 (2)]Z 1 - a 2 (2)Z 2 (19F.10) 
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Figure 19F-1. First-Order Spectra 

G (f ) =[ l+a^ 1) H-a^ (2)J - 2a 2 (2) cos (2nf/f N > 

- 2a 2 (l) [1- a 2 (2)] cos (irf/y (19F.1I) 

The zeros of Q(Z) now occur at: 

Z l,2 = 2 a 2 (1) ±2 ^2(D+4a 2 (2)] % (19F.12) 

An excellent example for a second order autoregressive time series is given 
by Fuller (1976, p. 55): 

x ( t) - x ( t - 1) + 0.89 * ( t - 2) =?(t) (19F.13) 

a 2 (l) = l a 2 (2)=-0.89 (19F.14) 


The zeros of Q(Z) now occur at: 



346 


Z1 2 = 0.500 + j 0.800 

y 

= 0.943 eij 1.01 

A plot of the locations of both zeros is shown in Figure 19F-2 
correlation function for this process has the appearance of an 
damped sinusoid. The power transfer function is: 

G (f) =2.79 + 1 . 78 COS (2-rrf/f^) - 3.78 COS (nf/^) 



Figure 19F-2. Zeros of Q(Z) =1- Z' 1 + 0.89 Z' 2 


(19F.15) 

The auto- 
exponentially 

(19F.16) 


Re[Z} 
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Figure 19F ,,3- Typical Spectral Peak for MEM 

A plot of Sx(f) = 'irs shown in Figure 19F-3. A multitude of the 
properties of MEM spectral estimators can now be summarized using the second 
order process as a typical example. These properties are: 

1. Q(Z) gives a complete description of the spectral estimate and the 
location of its zeros carry important information about the spec- 
tral shape. 
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2. Q(Z) is defined so that all of its zeros lie inside the unit 
circle. 

3. For second and higher orders, the zeros of Q(Z) may be complex. 

i 

Zeros lying close to the unit circle are capable of representing 
nearly periodic structure and thus allow MEM to estimate both 
noise and quasi-periodic components. Zeros off the real axis al- 
ways come in complex conjugate pairs. 

4. For each quasi-periodic component, two orders are needed for Q(Z) 
because the zeros must come in complex conjugate pairs. Thus to 
describe five quasi-periodic components the polynomial must be of 
at least order ten. 

5. The Burg recursion restricts the K-th autoregressive component to 

the range: °< a K< K > <1 thereby assuring a positive definite co- 

variance matrix and the absence of zeros lying exactly on the unit 
circle. 

6. The frequency of a zero that is sufficiently isolated from other 
zeros can be obtained approximately by: 

f M lm{Z } 

, p =-^A R CIA H {^|- } } (I9F.17) 

P 

For the Fuller example, f =- 0.322 f^. 

7. Some estimates can give a very sharp peak as illustrated by Figure 
19F-3. when designing a numerical algorithm for the estimator one 
must guarantee that no spectral peaks are missing. A quality 
check can be perforated by integrating S x (f) to see if it gives the 
total power. 
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When the approximate peak locations are known an expanded resolution as 
illustrated by Figure 19F-4 may be economical. 

Another approach to determining the location of spectral peaks is to 
determine the zeros of Q(Z) directly from a polynomial solving routine: 

Q(Z) = (Z -Z 1 )(Z-Z 2 )(Z-Z 3 ) ... (Z-Z^/Z* (19F.18) 

This works especially well if the zeros are well separated and the order is 
within reasonable limits. This approach also has the advantage that the 
stability of the zero locations and their separation from the unit circle 
can be more directly observed as K is increased. An example of this ap- 
proach will be given in Section H of this chapter. 



Figure 19F-4. Expanded Resolution of Spectral Peak 

Showing the General Shape to be Expected 
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G. MEM Resolution and Stability 

This section is essentially an extension of the last section with an 
emphasis on the resolution and stability properties of the MEM spectral 
estimate. The estimator given by Equations I5F.I or 19F.I is shown to be 
continuous, however, for a practical computer implementation, it is dis- 
crete. The practical display resolution, Af, is limited by the amount of 
computational effort one is willing to invest into Equation I5F.I. A dis- 
crete estimator will be represented by 


AtP. 


s x (nAf) = 


K 


K 


1-2 a v (k)e 
k=l K 


- j2TmAfkAT 


(19G.I) 


where: 

[n =0,+l, +2, .••,+( fjj/Af)} (19G.2) 

The analogy between MEM and digital filtering will be fully exploited 
in this section. To begin we redefine the complex polynomial to agree with 


1 ZK 

H(Z) =Q(Z) =D(Z) 

(19G.3) 

D(Z) =d K (0)+d K (l)Z + d K (2)Z 2 + ••• +d K (K)Z K 

(19G.4) 

D(Z) = (Z-Z 1 )(Z-Z 2 )(Z-Z 3 ) ... (Z-Z^ 

(19G.5) 

z _ e +j2TTfAT 

(19G.6) 


The coefficients of D(Z) can be identified with those of Q(Z) by the 
following: 
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^(K) = 1 ^(K. k) =-a K (k) (19G.7) 

The complex transfer function H(Z) represents the moving-average filter 
model. The zeros of Q(Z) are also the zeros of D(Z). The K-th order zero 
of H(Z) at zero represents a phase angle of KTTf/^ at f but has no effect 
on the spectrum. The mem power spectrum can also be determined by: 

Sx(f) =Atp k h(Z) H (Z) I (19G.8) 

2 = e +j2TTfAT 


It is common practice to represent each factor of Equation 19G.5 as an am- 
plitude and phase 


(z 


=r n e 


+i'eh 


(19G.9) 


so that (19G.5) can be expressed in an alternate form as: 

+j(8 + 6 + . . . + 0 ) 

D(Z) = . . . r R e z K (19G.IO) 


The magnitudes, r , represent the magnitudes of a vector from the zero 
location to a location on the unit circle and 9 K is the angle this vector 
makes with the abscissa. The filter transfer function can now be written as: 


H(Z) 


+j(KTTf/f N -e 1 -9 2 ... -e K ) 


r l r 2 r 3 * * * r K 


(19G.11) 


The spectral estimate becomes: 


s (f) 

x 


(r x r 2 r 3 . .. r R ) 


(19G.12) 
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The concept of a complex frequency location to describe the zeros of D(Z) 
will now be given. Its use will become apparent later. A complex frequency 
n is defined as 

n 


W 3o h 


(19G.13) 


and is used to describe the location of a zero of D(Z) in complex notation: 



n 


Z = e e 


(19G.14) 

(I9G.15) 


The factors in (19G.9) are expressed in terms of a complex frequency as: 


.(Z - ZJ =(Z - e 


(19G.16) 


On the unit circle we get 


(Z-Z„)=[e 


- e e ] 


(19G.17) 


or alternately: 


+jTTf/f^ -nor /f^j +jTT(B - f)/f* 
(Z - Z ) = e N [1 - — N p 


(I9G.18) 


The absolute magnitude of (19G.18) is: 

. -2-na I -TTnr / , 

IZ-Z n I =r n ={l +e n -2e n ^ cos (n(^ -f r (19G .19) 

The spectral shape of Equation 19G.12 is dominated by the function 

Iz -Z f near an isolated zero, Z n , that is close to the unit circle. The 
2 

function r Q is conceptually very important because it represents the generic 
shape factor of the mem spectral estimator near all isolated zeros of D(Z) 
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2 

when the zero is near the unit circle (small a ) . r is a function of both 

n n 

parameters, and g , and of the frequency variable f. This is the 

reason for the concept of a complex frequency to represent the zero. In 

general we can use (19G.12) to represent the spectral estimate with the 
2 

amplitude terms r n formally defined for each zero Z n as: 

2 -2170- /£^ -rror /f 

r (f ) = [1 + e n - 2e n - cos (iT(g - f) / f ) ] (19G.20) 


with: 


a = - - 
n tt 


to! IZ„ I } 


a > 0 


(19G.21) 



tt 


A rg(Z n } 


-f N ^ ^ +f N 


(19G.22) 


Consider the example shown in Figure 19G-I for a 3rd-order process 
with a zero on the real axis and a single complex conjugate pair. For fre- 
quencies very near tbe zero, Z3 1 we have nf/f^ =- 6 ^ and f =- 8 ^* The esti- 
mated spectral density is approximately: 


S (f) = 


atp k 

. -2770 /f -ttci' /£_ 

r l < ^3) r 2< ^3) Cl + e “-2e cos (77(83 - f)/^) ] 

(19G.23) 


For or the spectral peak is influenced only by the closest zero 

and occurs at f =$ 3 . The estimated spectral density has a peak of: 


ATP„ 


s (f) = 

x 


2,. , " 2TO 3 /£ N , -™3 /f N 1 

r 1 ^ 3 ^ r 2 ( ^*- 1 + e " 26 - 


(19G.24) 
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For a «f., the peak is symmetrical about f = B • I f we define the 
n N n 

frequencies &~+Af/2 as the points where S (f) is down from the peak value 

1 1 A 

by an amount k q then the bandwidth Af can be determined from the expression: 


+j I m [ Z } 



Figure 19G-I. Complex Plane Interpretation of an MEM 

Spectral Estimate for a 3rd-Order Process 


-2TTor /£, -•nor /£. 

[He n ^-2e n 1 

r ^(3 + A f /2) K o 

n n 

The solution for Af in terms of the cosine function is given by: 

cos( 2^ ) = 


(I9G.25) 


(I9G.26) 
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-va/L, 
= [ 1 + e a ^ 


lf N 

2e n N 


[1 + 


-2TTQ 1 / i 

e n 


-2 no? I f. T 
I +e n « 

”170? I £.. 

2e n « 


(19G.27) 


= ^ ARCC0S(K 2 [1- K q K-]} (19G.28) 

For small K 2 is very close to unity. As an example, take the solution 
represented by Equation 19F.15 and Figures 19F-3 and 19F-4. The appropri- 
ate factors are: 

a =- ^ {0.943} =- - n N (-0.0587) (19G.29) 

K 1= 1.72X10- 3 K 2 =“ 1.0017 

Assume a width where the spectrum is down by a factor of K q = 2 and the 
solution is: 


f N 


% 


= ~ ARCCOS (0.9983} =0.0374 


(19G.30) 


This result can be verified from Figure 19F-4. 

Next we consider the spectral resolving ability of MEM for two closely 
spaced- zeros near the unit circle. Assume they are symmetrically spaced 
about a nominal frequency such that: 


3, =B+Af/2 
I o 

P 2 = 8 0 - Af/2 

A 

Peaks occur in Sx(f) at f = 6 2 and a valley occurs at f = $ q . 


(19G.3I) 
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First assume the zeros are equidistant from the unit circle so = 
a^ = a Q . Then the ratio of the dip to one of the peaks can be written as: 


■>o> r 2 (6 o> 


-1 


= K_ 


r fo> r 2<V ° 


(19G.32) 


Equation 19G.32 can be evaluated using the results given by (19G.19) and 
the frequencies, = f and 3^ = 3 +Af/2. Expanding the terms gives: 

r l ' 2 "V • m ' £ N ,nA£ %1 2 

[ I +e 0 - 2e o cos 1 

.y y 

- 2 to I'L, -rro' I't, -2 to /'l. -to /' f f 

[l+e o « . 2e o ^][l + e ° - 2e 0 N cos 

TJ 


This results in a quadratic equation in cos(~^) given by 

“N 

tl + 2K Q K | ] cos^ - 2K^ cos (7^) 


^ c ' - ^l\') J 

N 2 2f N 

+ Ck 2 - k 0 k 1 (k 2 + 1) ] = 0 


(19G.33) 


(19G.34) 


where: 


Kl 


-2 to / 1, -to / £ -2to / f-. T 

= [l + e 0 ^ 2e 0 l+e 0 “ 

1 “ J£ “ 4 and K„ = " “ (19G.35) 

-TO 2 -TO / 

2e 0 ^ 


2e 0 




For two zeros spaced from the unit circle such that a =.01 (IZ I =-0.99) 
r n ' n 

and for a peak-to-val ley ratio K Q =2 the quadratic equation can be solved 
to give: 


jr “ 0.0556 
TI 


(19G.36) 
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The solution of (19G.34) must be done with high numerical precision because 
the coefficients are so close to a value corresponding to if=0. 

These equations illustrate the type of resolution, if, needed to evalu- 
ate Equation 19G.1 when the zero locations are known from evaluating the 
polynomials D(Z) or Q(Z). A check on the accuracy of § x (f) is to numerically 
integrate i t to be sure it is equal to the total power. If any peaks are 
unresolved the integration will yield a value less than the total power. 

A solution for resolving power when the zeros are not equidistant from 
the unit circle is much more difficult. Simplifying assumptions about the 
location of peaks and valleys are no longer valid. An estimator for the 
locations of the peaks and valleys as a function of if would have to be de- 
termined. This more general problem has not been solved. 

To determine stability and accuracy of the estimate we need to relate 
the data to the computation of the zeros of D(Z). This will be done for a 
second-order process. For a second-order process the polynomial is 

D(Z) =-a 2 (2) -a 2 (l)Z + Z 2 (I9G.37) 

and the zero locations are: 

Z l,2 = 2 a 2 (1) ± 2 i>2 (1) + 4a 2 (2)]% (19G.38) 

If the roots are complex (a 2 (2) <0 and |4a 2 (2) | >a 2 (l) the solutions are: 

Z l,2 = \ a 2 (1) ±T jC4|a 2 ( 2 )|-a 2 (1 )] % (I9G.39) 

Examples of second-order processes can be found in many texts. A good ref- 
erence is Fuller (1976, p. 54). 
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For complex roots the following solutions are obtained for the root 
locations and the sampled autocorrelation function of the process. The 
absolute values of the roots are: 


^1= |Z 2 |= U 2 (2)| % (19G.40) 

The sampled autocorrelation function for positive k is (Fuller, 1976): 

R (kAT) = A(kAT) sin [tr kAT + <2 ] (19G.4I) 

X 

Cos(^) = ^ a 2 ( I ) I a 2 (2) \ h (19G.42) 


i-a,(2) 

T an(a) = LYT^y] tan (~) 

k(2>i k/2 ^n^kAT 
A(k * T) “Sin* 

From the concept of complex frequency we identify: 


(19G.43) 


(19G.44) 


a - - ^ £n{|a 2 (2)l % } (19G.45) 

3 = ^ARCCOS{| a 2 (l)|a 2 (2)i % } (19G.46) 


Keeping these results in mind we can now consider the Burg algorithm 
for MEM and possible sampling effects. First of all we know that v® is 
determined by minimizing the error of Mj, (Equation I5G.6) in a least squares 
sense. We also know that the total amount of data used for the estimate 
(Equation I5G.8) decreases with increasing K. This means that the variance 
of ^(K) for a given set of data i ncreases with increasing K. Alternately 



359 


for a fixed value of K (K =2 in our example) the variance decreases with 
increasing amounts of data. Also as the data increases the variance of the 
estimated error P K (Equation I5G.4) decreases. The general trends of these 
varfances are shown in Figure 19G-2. The concept of variance for ^(K) can 
be shown using Equation (I5G.I2) and defining an ensemble with index N to 
illustrate dependence on sample length: 

VarU K (K,N)} = E{a^(K,N)}^0 N < <° (I9G.47) 

For large amounts of data the expected values of P K and ^(K) tend to 
zero for large K. For smaller data sets they tend to decrease until the 
mean is masked by the variability. For some maximum K =K : 

E { a K m (K m ) } "0 (19G.48) 

E[Pi/ } =P K -1[1- E 2 {a^ (KJ} - P K -1 (19G.49) 

K m K m m 

Typical trends are shown in Figures 19G-3 and 19G-4. For actual data 
records the masking effect is shown in 19G-5. 

The variability of |a^(K)| directly affects the distance between a 
zero and the unit circle (Equation 19G.45) thus affecting the peak value. 

In practical applications this can be seen as a change in peak value for 
increasing K and as the data record changes. 

MEM is becoming more widely known and used because it is an estimator 
which has better spectral resolution than the DFT for short data records. 
This effect can be explained for certain types of common sampling situations. 
The two most common examples are: severe oversampling of short data records 



VarCa^K.N)) Varfa^K,!*)} 
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I ncreasi ng K 


K=N 



I ncreasi ng N 


Figure 19G-2. 


General Trends of the Variability of 
a_(K) for Fixed Data Records and for 
Variable Data Records 
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Increasing K n 

Figure 19G-5. Typical Trend for Actual Estimates of the 
Total Mean-Square Error of the One-Step 
Prediction 

with one periodic component and a reasonable amount of bandlimited noise or 
long observation periods with a moderate sampling rate and bandlimited 
noise. 

Since the autoregressive coefficients can implement both a one-step 
predictor in time (Equation 13.35) a'S well as autocorrelation (Equation 
14.42) and since the coefficients are determined by minimizing the mean- 
square error (Equation I5G.6) and are not explicitly dependent on the 
sampling interval (&t = AT), the estimate can be considerably improved by 
oversampling. Not only does it decrease the variance of ^(K) due to noise 
but once the data record is long enough to include more than about 60% of 
a cycle of the sinusoid represented by Equation 19G.41 the extrapolation 



property takes over and produces a strong periodic component. The extra- 
polation is helped by oversampling. For longer records the sampling prop- 
erties are analogous to those discussed in Chapter XII Section D, Part 4. 
In effect a single period of ft (T) is oversampled and the extrapolation 
property completes the task of producing a sharp peak. 

The effect of increasing the order of the estimate on the spectral 
resolution is less obvious. For a given fixed record length, increasing 
the order increases the nuaber of zeros of D(Z). Now only zeros close to 
the unit circle produce a significant effect on the spectral shape so for 
given periodicities in the data only these zeros are needed. For a finite 
and noisy record however these zeros are required to represent a least- 
squares estimate for. all of the data. The net effect is to shift the 
zeros away from the unit circle. By increasing the order, K, of the esti- 
mate, additional zeros are added (additional degrees of freedom for the 
prediction) that are far from the unit circle thereby having little effect 
on shape and at the same time allowing the important zeros to move closer 
to the unit circle. For this reason the spectral resolution is improved 
(sharper peaks) and the location of the peak is more accurate. The resolu- 
tion of two closely spaced and equal peaks is a function only of the dis- 
tance between their zeros and the unit circle. Figure 19G-6 illustrates a 
spectral representation of the effect of zero location. 

A particular zero, Z , has a random distribution in a circular sense 

around some nominal location. The distribution for Iz lisa cross section 

n 

of this circular distribution. When Z is close to the unit circle a small 

n 

change in location has a very gross effect on the amplitude and location of 
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the peak. This shifting and variability can be seen for any practical 
record. A point of diminishing returns is reached where the average zero 
location is no longer moved closer to the unit circle by increasing the 
order, K. The order should not be increased beyond this peint. A pictorial 
representation of this effect for an isolated zero near the unit circle is 
shown in Figure 19G-7. 



Figure 19G-7. V ari abi I ity of the Estimated Zero L ocation 
for Increasing Orders of the Estimate 
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H. DFT and MEM Resolution Limits 

1. I ntroduction 

When considering the resolution of a spectral estimator, one must be 
careful to distinguish between the ability to detect and resolve periodic 
components and the frequency resolution to which the resulting estimate is 
displayed. The latter will be referred to as the display resolution while 
the former will be called the spectral resol uti on or simply the resolution 
of the estimator. 

An increase in display resol ution is only a matter of increasing 
processor time. It makes the estimate appear more continuous and may be 
used to produce a more pleasing appearance. High display resolution is 
particularly needed for MEM because a sharp peak may otherwise be over- 
looked. For the FFT, higher display resolution may sometimes improve the 
frequency ambiguity of a spectral peak. 

In contrast, the spectral resolution of the estimator is an inherent 
property that is determined by the "quality" of the estimator and the ob- 
servation time for the data. . It cannot be improved by postprocessing 
techniques. 

2. D isplay resolution 

For MEM spectral analysis the information about sharp spectral peaks 
is all contained in the zero locations of the complex polynomial, D(Z). 
These sharp peaks will always be displayed if the frequency interval 
(Equation 19G.I) is kept small enough. For a given polynomial, the dis- 
play resolution can be arbitrarily increased by decreasing Af without af- 
fecting estimator quality. 
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For the FFT, the display resolution problem caused by a sampled spec- 
trum can be minimized by either using a continuous estimator (Equation 
11.1) or by concatenation of zeros to the record. Zero concatenation is a 
well-established technique in FFT analysis and is often used to make the 
number of data points the required power-of-two. Concatenation forces the 
FFT to treat the data as though it were a periodic time pulse with a low 
duty cycle. For large orders of concatenation, the spectral estimate 
closely approximates that for a single time pulse. Concatenation also in- 
creases the apparent observation time thereby causing the apparent ampli- 
tudes and frequencies to shift slightly. This can sometimes result in a 
slightly better estimate of the frequency of peak amplitude. The disad- 
vantage to large order concatenation is increased processing time. A 
simple dft does not require zero concatenation. 


3. Spectral resolution 

The fundamental resolution limit for the dft is imposed by TN and was 
given for two closely spaced sinusoids by Equation I2B.48. The Rayleigh 
resolution or one-half the bandwidth-between-first-nulls is given by: 



(I9H .1) 


This resolution limit assumed a continuous estimator (Equation 11.1) and 
peaks of equal amplitude. Neither assumption is the case in most practical 
FFT applications. In most FFT analysis, the estimated location-of-peak is 
within ± Hz. of the true peak. Zero concatenation does not improve this 
resolution. 
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Improvements in the spectral resolution of the DFT can only be ob- 
tained by increasing the observation time, TNo 

The spectral resolution for the mem spectral estimator is limited by 
the number of independent samples of the time series (observation time, TN) 
and the degrees-of-freedom for the spectral estimator (order of the esti- 
mated autoregression)o Increasing the number of samples decreases the 
variance of the estimate of the expected values of the autoregressive co- 
efficients. Increasing the order of the estimate allows more degrees-of- 
freedom to approximate higher order regressions thereby minimizing the 
error associated with the most prominent spectral peaks. This choice of 
the degrees-of-freedom is contrasted with the situation for the DFT where 
the order i s -fixed. 

Mathematical derivations for MEM resolution as a function of the 
number of samples and the order of the estimate have yet to be published. 

4. HEM resolution I i mits 

The resolution limit for MEM spectral analysis in terms of the number 
of samples and the order of the estimate is much more difficult to assess 
than for the DFT. For the latter, the degrees-of-freedom are fixed due to 
the periodic assumption and hence for each two time samples there is an 
additional pair of orthogonal components. Also, there is no least-mean- 
square error to be minimized. In MEM the degrees-of-freedom are selected 
by the user and hence the resolution limit is affected both by the number 
of samples and the order of the estimate. 

The presentation of mem resolution in Section G of this chapter ad- 
dresses the properties associated with the location of the zeros of D(Z) 
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but does not present the effects of the number of samples or the order. 

Some of the present MEM literature refers to the assumption that MEM uses 
all of the data at each step in the recursive algorithm. It does not! End 
effects, although minimized in the Burg algorithm (Equations in Chapter xv. 
Section g) , are not eliminated for higher order autocorrelation components. 
Reduction in the amount of data used is clearly evident in the summation 
index of Equation I5G.12. End effects cause an increase in the variance of 
the autoregressive coefficients as the order is increased (Figure 19G-2) 
because of the reduced amounts of data used in the estimate. 

Parzen (1969) has suggested, but did not prove, a method for deter- 
mining the order based on the variance of the least-mean-square error esti- 
mate. Several examples examined by the author show that this technique 
produces an estimate that does not have the best resolution inherent in 
the estimator. A better criteria is the stability (or variance) of the 
location of a critical zero of D(Z). 

A general rule is to choose a sample length that will span more than 
one-half of the period of the lowest frequency component and chose an order 
high enough so that the variance of the zero locations of spectral peaks 
of interest is minimized. A detailed example will be given at the end of 
this section. 

5. DFI resolution limi t 

From an estimation theory viewpoint, the resolution limit for the DFT 
is given by Equation 19H.1. It states simply that the resolution is limited 
by. the observation time, T^. From a theoretical viewpoint, the exact effect 
of DFT sampling on the estimated amplitude spectrum of a periodic function 
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is specified by the estimated Kronecker delta function defined in Equation 
1 2 D .11 and used in Equation I2D.10. 

Figure 11-1 illustrates the bandpass or weighting effect represented 
by Hrq(N ,T , Tp) for a single spectral component. The weighting function 
is specified for each harmonic number or frequency. The true amplitude of 
the spectral component is slightly reduced with the corresponding power 
distributed to other frequencies. The resolution limit for smaller ampli- 
tudes is limited by the peak value of spurious components. The magnitudes 
of the spurious components can be related to the "parent" peak by using 
the spectral mixing formula. 

For a large peak at frequency f L , the principal and spurious complex 
amplitudes can be obtained from a single term of the series in Equation 
11 . 20 : 

c(f L’T^ n) =C L^2N+ 1 ^ 1+2 k f 1 C ° S(2TTf L _ ^ )k 2N+1^ (19H.2) 

For estimation purposes, c L is replaced by its estimate when calculating 
spurious components. An example of this effect is illustrated in Figure 
I2D-3. For T^ =3.1, TF=I.O, and (2N+I)=2I, the pri nci pal component and 
frequency are: 

|cj =2.0 f L =0.968 Hz 

The nearest spurious component has an amplitude and frequency of: 


1 ^ 1 = 0.186 


£ Lfl=l-29 Hz 
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A small adjacent spectral component could not be resolved if its amplitude 
did not exceed I cL +11. 

The location of a spectral peak near zero frequency can be masked by 
the average value if only a fraction of a cycle is sampled. This type of 
error is illustrated in Figure I2D-2. Since partial cycles cannot be re- 
solved using the DFT it is highly desirable to remove the average value 
before processing. 

6. Example for MEM 

The interferometer data used as an example in Sections D, F, and G of 
Chapter XVII is also an excellent practical example for illustrating the 
effect of filter order on the MEM spectral estimator. The output from 
PROGRAM- 07 in Appendix III provides the data used in this example. 

The time series (HER A 7/29/75) was processed using a variable filter 
length, K =1 - 15. The estimated autoregressive coefficients are printed 
for each order of the estimate. Figure 19H-I illustrates the variability 
of the estimated coefficients as a function of the chosen filter length for 
K =1 - 9. The variation of the magnitude of these estimates directly affects 
the zero locations of the z-transform polynomial they generate. The most 
variable coefficient is ^(K) because it is the independent variable for 
the one-step prediction. 

The estimated coefficients were used to generate the complex poly- 
nomial, D (Z), as described in Equations 19G.4 and 19G.5 and the roots were 
obtained using a polynomial root solver. A summary of the results for 
filter orders up to ten is shown in Table 19H-1. Each root is described 
by a top entry which is the magnitude (iz 1<1) and a bottom entry which is 
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the phase angle in radians (-11 £ Arg{z n } £+rr) . These root locations are 
plotted on the unit circle as shown in Figure 19H-2. The zeros represent- 
ing the principal peak are labled Zl and are marked with a dotted line. 



Figure 19H-1. V ari abi I ity of the Estimated A utoregressive 
Coefficients as a Function of the Order, K, 
of the Estimate 


The estimated location-of-spectral-peak for Zl varies as a function 
of the order K. Figure 19H-3 illustrates this variation. This character- 


ion G and an example plot given in Figure 19G-7. 


Table 19H-I. Zeros of the Oomplex Polynomial, D(Z) 


ORDER 

K 

Z1 

h 

Z3 

Z4 

Zs 

Z6 

11 

Z 

+0.9229 








±.687 







3 

+0.9446 

mm 




+0.2008 



±.67S 





+ TT 


4 

+0.9411 

mm 

— — 


mm 

+0.3824 

+0.2100 


±.684 





+TT 

0 

5 

+0.9572 

+0.6275 

mm 


_ _ 

mm 

+0.6539 


±.707 

±2.30 





0 

6 

+0.9267 

+0.5798 

mm 


mm 

+0.2777 

+0.7149 


±.707 

±2.16 




+ TT 

0 

7 

+0.9681 

+0.6608 

+0.5255 

mm 

mm 

mm 

+0.7861 


±.702 

±2.47 

+1,55 




0 

8 

+0.9718 

+0.8616 

+0.8283 

+0.8128 





±.725 

±2.60 

±L 71 

±.349 




9 

+0.9816 

+0.8262 

+0.8554 

+0.8893 

mm 
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+0.8302 
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+0.4966 




±.725 

±2.58 

±1.56 

±,337 

±2,25 
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Figure 19H-3. Migration of the Estimated Location of 
the Spectral Peak Represented by Zl 

Several interesting features of the MEM spectral estimator are illus- 
trated in this example. The first thing we realize is that the mean-square 
estimation error is not a suitable measure for selecting the order of the 
estimate. From Figure I7G-5 we see that the error decreases very little 
beyond order K =2 and yet Figure 19H-3 shows the estimated location-of- 
spectral peak improving up to order K =15. This improvement comes about 
because the degrees-of-f reedom were increased thereby allowing the zero 
location to migrate to its preferred position. Also the peak became 
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sharper as indicated by the magnitude of z, approaching unity. The impor- 
tant conclusion is that the zero location is a better indicator of the 
desired order of the estimate. 

Figure 19H-2 shows the effect of increasing order on all zero loca- 
tions. It is important to note that each spectral peak to be resolved 
requires two additional orders of the polynomial. This is illustrated for 
order K=5 where two peaks are modeled. In other words, only those zeros 
in the upper-half-plane have a significant effect on spectral shape. From 
Table 19H-I and Figure 19H-2 we also see that the identity of the most 
important zeros can be preserved over many orders of the estimate. This 
property also helps in the selection of a maximum order. 

The techniques of analysis presented in this example are typical of 
those that may be used for practical MEM spectral estimation. For each 
prominent spectral peak a separate plot similar to Figure 19H-3 should be 
made. The "true" location is marked for reference and illustration. In 
general an estimate of the true location must be made. Estimates of the 
total power in a spectral peak must be obtained by computing the area under 
the peak. This power spectrum technique is in direct contrast to FFT anal- 
ysis where the amplitude of the peak is important. 

I. Review of Common Spectral Estimation Errors 
1. I ntroduction 

This review of some of the more common errors associated with the es- 
timation of amplitude and power spectra will specifically address the DFT 
(also fft) and mem. The techniques and effects of data preprocessing on 
errors have been discussed in Chapter IX, Section C. H ere we concentrate 
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on the major effects of observer bias and error and intrinsic estimation 
errors. Gross observer error, equipment malfunction error and lack of 
stationarity will not be addressed. 

2. A liasing 

Aliasing or the frequency fold-over effect caused by sampling at a 
rate less than the Nyquist rate has been discussed in Chapters VI and IX. 

In the FFT literature, aliasing has been given considerable attention while 
in MEM literature it is seldom mentioned. The reader should be keenly 
aware that it can occur in both types of analyses. 

The effects of aliasing can be minimized or eliminated by careful ex- 
perimental design such as prefiltering ahead of the sampler. Under certain 
special circumstances aliasing is desirable. An example of a practical 
application of aliasing is for the detection of a sinewave in a spectrum 
much wider than the principal alias. The fold-over effect allows the sinu- 
soid to be detected without an increase in the sampling rate. 

For the vast majority of spectral analysis applications aliasing must 
be eliminated and, when reporting results, it is helpful to inform the 
reader as to how this was accomplished. 

3. N umerical stabi I ity and accuracy 

Because the word length is finite for digital processing systems, it 
is possible to introduce noise and instabilities into the various spectral 
estimation algorithms. Truncation or roundoff error will introduce wide- 
band noise into the estimate. 
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Truncation can be serious for MEM because, for good signal-to-noise 
ratios, it can cause a zero of the characteristic polynomial, D(Z), to lie 
on the unit circle. Some applications may require the use of double pre- 
cision or the addition of a small amount of wideband noise to eliminate 
this effect. A well designed MEM algorithm should check for this effect. 

For a majority of practical scientific data processing, numerical 
stability and accuracy will not be a problem. Only for the cases of very 
high predictability or very small word length should the user apply special 
precautions. 

4. Spectral smoothing 

Spectral smoothing or the loss of spectral resolution caused by a 
finite observation time is discussed in Chapter IX. Smoothing in FFT anal- 
ysis is caused by a finite time window. In the B lackman-Tukey method it is 
caused by a finite lag window. Smoothing in MEM analysis is the result of 
using a smoothed estimate for extrapolating the autocorrelation function 
and using a single zero of the complex polynomial, D(Z), to represent a 
single spectral peak. 

The undesirable effect of spectral smoothing can only be reduced by 
increasing the number of samples (or the observation time). For MEM, im- 
proved spectral resolution (over FFT) is the result of using an estimator 
that has a selectable degrees-of-freedom. 
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5. Spectral leakage and end effects 

Spectral leakage (or spectral mixing) is caused by the finite obser- 
vation time (finite time window) and is discussed in Chapter IX. The spec- 
tral mixing formula derived in Chapter XI is an exact model for represent- 
ing the affect of a specific DFT estimate on spectral leakage. For mem, 
spectral leakage is observed as a shift in the absolute magnitude of the 
zeros representing spectral peaks. Leakage reduces the ability to resolve 
minor components because a large -peak may have leakage that obscures a 
minor one. For DFT analysis the spectral mixing formula can be used to 
place an upper bound on leakage and hence on minor component detection. 

For MEM, no technique, as yet, exists. 

The variability of estimated autocorrelations with increasing lags is 
caused by the end effect of finite data records. For the B lackman-Tukey 
method, end effects are minimized by the proper choice of a lag window 
(Blackman and Tukey, 1958). For MEM, end effects are minimized by the 
least-squares smoothing used in extrapolating the autocorrelation function. 
End effects for MEM are further discussed in Section G of Chapter XV. 

6. N oise and stati sti cal vari abi I itv 

Sample records of the time series representing a physical process are 
usually contaminated by random noise inherent in the measurement process. 
Also, since the sample records are finite, the estimated means of the 
various parameters have a considerable variability that depends on the 
length of the sample. These effects introduce random noise and statistical 
variability into the various spectral estimators. 
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The effect of noise on the DFT estimate was discussed in Section D of 
this chapter. Statistical variability due to sampling effects can be 
analysed using the spectral mixing model and Rayleigh statistics. 

For mem, measurement noise is modeled as Gaussian noise with a band- 
limited spectrum. In this model, all noise is treated equally so measure- 
ment noise and random processes effects are indistinguishable. Statistical 
variability appears in the estimated autoregressive coefficients and is 
caused by noise and sampling effects. Variability is reduced by taking 
longer sample records. 

Spectrum averaging is frequently used in FFT analysis to reduce the 
effects of statistical variability. In MEM, spectrum averaging and in- 
creased sample length are both helpful. 

7. F ractional period sampling 

All spectral estimators perform poorly if only a fraction of the period 
of a spectral peak is sampled. For DFT (and FFT) analysis, samples less 
than 0.9 Tp are unusable as most of the amplitude will appear as a zero 
frequency component. For all DFT applications, the data taking method must 
be designed so at least nine-tenths of the period of the lowest frequency 
component is sampled. 

The MEM estimator can give good results when only 60% of a period is 
sampled. This requires severe oversampling and a reasonable low noise 
variance. MEM performance is discussed further in Section G of this 
chapter. 
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8. Spectral strength or magnitude 

Interpretation of the results of a spectral estimate most often in- 
volves the selection of the "most prominent" spectral component. For DFT 
(and FFT) analysis this simply means choosing the highest peak because the 
height is proportional to the amplitude of a sinusoid at that frequency. 

For MEM, the estimate is a power spectral density and the amplitude of 
a periodic component is proportional to the area under the peak. When 
analyzing MEM records, one should avoid the temptation of looking at the 
heights of the components and instead compute the areas for comparison. 

9. L ocati on- of -spectral- peak (LOSP) 

The location or estimated frequency of a spectral peak is another 
important property of a spectral estimator. For FFT analysis, the LOSP is 
affected by the number of samples and the record length and by zero con- 
catenation. If only a fraction of a period of the lowest spectral component 
is sampled, the LOSP for that component will occur at zero frequency. The 
LOSP for MEM analysis is affected by the number of samples and the order of 
the estimate. Typical effects where discussed in Sections G and H of this 
chapter. 

The user must realize that the LOSP is statistically variable from 
record to record and take this into account when reporting results. If 
many records are available it is possible to get an estimate of the vari- 
ance of the LOSP. 
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10. MEM display resol ution 

Since the MEM spectral estimator (Equation I5F.I) is continuous with 
frequency, a digital implementation requires the selection of a display 
resolution, Af. For a very sharp peak it is possible that the output will 
not show the peak because it falls between frequency samples. For this 
reason a practical MEM program must contain a check for missed spectral 

A 

peaks. One way is to numerically integrate s (Af) to see if the total 
power is recovered. Another way is to compute the zero locations for D(Z) 
to see if one lies extremely close to the unit circle. MEM display resolu- 
tion is a practical problem that must always be accounted for by the user. 
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XXI. APPENDIX I - MATHEMATICAL TECHNIQUES 
A. Integration and Riemann Summation 


A function f(x) is said to be Riemann integrable if in the limit 

n b-a h -a 

l b = i f(x)dx = lim , s,f(a+k - - a (A 1 .1 ) 

a J a n — k=l n n 

where the interval a - b is divided into n segments with each segment having 
a width of (b - a)/n. If the number of segments, n, is not taken to the 
limit, the Riemann sum. will yield an estimate of the definite integral lab. 
The actual value of the estimate depends upon how many seqments are used 
and how the sum. is taken. For small n, a good technique of estimation is 
to evaluate the function at the midpoint of the segment for use in the 
summation. The integral estimate using this procedure becomes: 



b - a 
n 




(A 1 .2) 


Figure Al-I illustrates this procedure. 

For the purposes of this research work it was discovered that a 
specialized form of the Reimann sum approximation was very well-suited to 
Fourier series analysis. If we assume that f(x) is shifted so that the 
origin is at the midpoint of the independent variable, and that there are 
an odd number of segments, the definite integral 


aa 


f (x)dx 


(A i .3) 


-a 


can be estimated with the following sums 


+N 


f aa ‘ k»5( f(k4x)43t 


(A I .4) 
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Figure A I -I . Riemann Sum A pproximation for a 
Definite I ntegral 


where: 


f 


28 


s 

k»-N 


2 a 


aa- (2N+I) . f(k 


(AI.S) 


k = 0, + l, + 2,...+N. 

n = (2N + 1), the number of segments 


assumed odd. 


2a 2a 
& =~ = (2N +1) 


Figure A I - 2 illustrates this procedure. 

This particular estimator for the definite integral is used many times 
throughout this report. Its usefulness is illustrated in several of the 
derivations involving the estimation of the complex Fourier coefficients 
for digital Fourier transforms. 
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Figure A I - 2 . R iemann Sum. A pproxi mation used in 
Fourier Series Analysis 

B . H i I bert Space V ectors 

An infinite-dimensional vector space is called a "Hilbert space" and 
is denoted symbolically by W- In the "vectors" are not directed line 
segments, but rather are complex functions of real variables. We define 
a "vector" in to be a complex function, of a real variable x. In 
other words, a "vector 1 in^J is a rule of correspondence which assigns to 
each number x, a complex manner Kx). The vector space is defined as 
fol lows: 

a) Scalars are tbe set of all complex numbers, c. 

b) The distributive lav of addition holds (vector addition): 

♦00 - Cjtjte) + c 2 ijr 2 (x) 


(AI.G) 
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c) The vector inner product is defined to be 


= f ^*(x)t 2 (x)dx (A 1.7) 

all x 

with the following properties: 

(^ 2 » V = *W* (AL8) 

(c lV°2V = C 1 C 2 ( VV (AL9) 

|<W ! « (V V*( W* (ALI0) 

d) The vector norm is defined as 

(M) = S |*(x).l 2 dx (A 1 . 11) 

al I x 


and a function ty(x) must have a finite norm, < «, to be a 

vector i n 

e) A set of ^(-vectors £a^(x)} form a vector basis set in V. 
The set is said to be complete if any vector in ^ can be con- 
structed from a linear combination of the basis vectors {<y (x)}. 
The basis set is orthogonal if the inner product (a^,a^ is zero 
for all n#m. The basis set is orthonormal if ( 0 ' n , o^j) = 6^ 
(the Kronecker delta). 

f) An orthonormal set of eigenvectors {cr (x)} are said to form 
an orthonormal basis set in if the set is both orthonormal 
and complete. Any given ^-vector, ty(x), can be written as an 
infinite-dimensional (infinite series) linear combination of the 


orthonormal basis set 
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CD CD 

iir(x) = JEa n ar n (jO = (A1.12) 

where the a n ' s are the complex expansion coefficients (eigen- 
values of ijr) defined by the inner product: 

a n = (of ,t) = J or*(x) i)f(x)dx (A1.13) 

all x 

The vector norm of i}r(x) can be written in terms of the expansion 
coefficients as: 

09 2 

= S la | (A1.14) 

n=l 

The theory of Hilbert space vectors was successfully applied to Fourier 
transform theory to give many of the important derivations shown in the 
text. This formulation also can provide useful insight into the problem of 
estimating the complex Fourier coefficients. 

C. Deterministic Functions 

Mathematical functions which are made up of parameters which are com- 
pletely specified for all time and all values of the independent variables 
will be called deterministic. This terminology is used to denote that 
there is no "randomness” associated with the function. For practical pur- 
poses, a function of time x(t) is deterministic if it can be exactly charac- 
terized by a Fourier series. It is philosophically satisfying to realize 
that an observable function of time, x(t), must be periodic and therefore 
represented by a Fourier series if it is to be deterministic. Time funo- 
tions which are not periodic cannot be completely specified because an 
infinite observation time is required. 
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Deterministic time functions are periodic in the time domain and ex- 
hibit only line" spectra in the frequency domain. Deterministic functions 
cannot have continuous spectra. A pathological exception to this defini- 
tion exists for single pulses in time. These functions are known for all 
time and have an implied period which is infinite. They also are charac- 
terized by infinite and continuous power spectra. 

D. Noise and Random Processes 

A random process will be defined to be a collection of time functions 
and an associated complete probability description. The entire collection 
of time functions x frj is called an ensemble M random time functions of 
which the nth function is called a sample function of the process. In 
general, only one sample function of a random process can ever be observed; 
the other (n- 1) sample functions represent all other possible realizations 
which might have occurred but didn't. The following properties of random 
processes are important for our purposes: 
a) Continuous -- Discrete 

In a continuous random process, the random variables xn(t^) at 
time can assume any value within a specified range of values. The 
probability density function is continuous and has no Dirac delta 
functions in it. A discrete random process is one in which the random 
variables can assume only certain isolated values. The probability 
density function contains only Dirac delta functions and does not have 
a continuous component. A mixed random process is one in which 
probability density function may have both discrete and continuous 
properties. 
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b) Deterministic -- N ondetermi nistic 

If future values of x ( t) cannot be exactly predicted from ob- 
served past values, the process is called nondeterministic. If future 
values can be completely specified, the process is called determinis- 
tic. It is possible for a function to be random over the ensemble but 
not random with respect to time. An example of this type of process 
is an ensemble of sinewaves, x n (t), each with a different amplitude, 

A , where A R is a random variable. Any sample function of this pro- 
cess would be deterministic with respect to time. 

c) Stationary -- Nonstationary 

If all of the marginal and joint density functions describing a 
random process do not depend upon the choice of time origin, the 
process is said to be stationary. If these density functions do de- 
pend 1 upon time, the process is called nonstationarv. Most real physi- 
cal processes are nonstationary to some degree but quite often they 
can be characterized as stationary for practical considerations. 

A process is said to be stationary j_n the wide sense (or covari- 
ance stationary) when the expectation value, E{x(t|)}, is independent 
of the choi.ce of time origin and when E (x(t| jx^)} depends only upon 
the time difference, ( t2 - t|) =t. This guarantees that the mean value, 
mean-square, value, variance, and correlation coefficient are all inde- 
pendent of the choice of time origin. 

d) Ergodic -- Nonergodic 

A random process is said to be strictly ergodic if the time aver- 
age of one sample function is equal to the ensemble average. It is 
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ergodic with respect to its correlation function when the ensemble 
correlation function is equal to the time autocorrelation function. 

For ergodic processes, mean value and higher order moments can 
be determined by time averages over an infinite interval as well as 
by ensemble averages: 

+® 1 +T/2 

Efx 11 } = f x n p(x)dx = lim — f x n (t)dt (AI.IS) 

r. J T/2 

In this report, most random processes will be continuous, nondetermin- 
istic, stationary, and ergodic. a sample function of the process will 
be called "noise" or a random time function. Usually, only one sample 
function will be available for analysis and its statistical properties 
will be estimates of the "true" values. 

E. Correlation Functions for a Random Process 


The autocorrelation function Ry(t| ,t 2 ) of a random process is defined 
as the joint moment of the random variables y(tl) and y(.t 2 ): 


, +® 


,+® 


E{Y(t| )Y (t 2 ) } * MW = J dYI J Y I Y 2P ( Y I'Y 2 ) dY 2 (A 1 .16) 

* -Leo • as. 

The autocovariance. Cy ( t| ,t 2 ) of the process is defined as: 


EtfYO^) - Y(t x ) ][Y(t 2 ) -Y(t 2 )]} = C (t x ,t 2 ) 


(A 1.17) 


For a "wide-sense" stationary process, the ensemble averages defined 
above are 1 independent of the choice of time origin and the autocorrelation 
function becomes 


R y (T) * E{Y(t)Y(t+T)} 


(A 1.18) 
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where t = (t 2 - t^) is called the "lag time". For stationary processes, 
the time autocorrel ati on f uncti on and ti me autocovari ance functi on are 
defined as 


1 | + 7/2 

R (t) -lim - I y(t)y(t + T)dt (A 1.19) 

y T->® 1 -7/2 


and: 


1 +7/2 

C (t) = lim T J [y(t) - yet) J [y(t + t) - yet + t) ]dt (A 1 .20) 
y T-» -7/2 


For most of the work in this report we will let x(t) represent a ran- 


dom variable with zero mean and then define a zero-mean autocovari ance 


function as: 


1 1+7/2 

C (t) = lim -J x(t)x(t + T)dt ( A 1 . 2 1 ) 

X T-'® -7/2 


The autocovariance function is very important in the study of random pro- 


cesses and random time functions because it is used to define the power 


spectrum of the process. 


F. The Power Spectrum of a Random Process 

The power spectrum of a random process is defined as the Fourier 
transform of the autocorrelation function. For a wide-sense stationary 
random process the autocorrelation function is 

1 1 + 7/2 

R (t) = lim, T I x(t)x(t + T )dt (AI.22) 

x T-® '7/2 

where x(t) is a sample function of the process. The autocorrelation func- 
tion and the power spectral density function, S v ( ui> , (also called the 

X 
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second moment spectral density) are related by the Wiener Khinchine rela- 
tions (Cooper and McGillem, 1971, p. 148): 

S (o>)= J* R (T)e“ jUn d T (A 1.23) 

X -eo X 

| fp 

R x (T)= fcj S x (u))e +jujT daj (A 1.24) 

—00 

In theory, a knowledge of the autocorrelation function is equivalent 
to a knowledge of the power spectral density function. 

G. Autoregressive-Moving Average Processes 

Stochastic processes which generate a random time series can be suc- 
cessfully modeled as moving average processes, finit;e autoregressive pro- 
cesses, or autoregressive-moving average processes (Koopmans, 1974, Ch. 7). 
These models can then be used to generate an infinite tiae series, a dis- 
crete autocovariance function, and a continuous power spectrum. This type 
of analysis is widely used in statistics and is becoming increasingly 
important in engineering for studying the spectra of stochastic processes 
using the maximum entropy method. 

A f i ni te moving average process is modeled by the following series 
M 

x (t) = S (t- k) (A1.25) 

k— M 

where 2M+1 is the order of the series and the a^is are real constants. 

The sequence of uncorrelated random vari abl es, §(t): t*0, ±1, •••» models 
a white noise process. The cOIIIDOn mean for this sequence is E{|(t)}=0 
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2 2 

and the variance E{| (t)} = a . The finite moving average model can be 
i nterpretated as the weighted sum of 2M + 1 random and completely indepen- 
dent impulses, §(t), occuring at past and future times which gives the 
value x(t) at time t. These impulses (called random shocks by Koopmans) 
are most commonly referred to as i nnovati ons at time t (Parzen, 1969). 

If we impose the requirement of causality and oonsider models which depend 
only upon present and past innovations, we obtain a one-sided moving 
average: 

2M 

x(t) = k | 0 (t- k) (A 1.26) 


The model for a moving average process has vide applicability. In fact, 
every weakly stationary (covariance stationary) process that has a strictly 
continuous spectrum can be approximated arbitrarily close (in a mean- 
square sense) by. a finite moving average process. Kaopmans (1974, p. 214) 
shows that every weakly stationary process with a continuous spectrum has 
an infinite moving average representation. 

The scheme of I i near autorearessi on i sa technique that is used to 
approximate the value of a given finite series at a time t by soae 
weighted sum of past values of the series. Finite autoregressive proc- 
esses are a class of finite parameter models that may be represented by 
this approach. The finite order autoregression satisfies the equation: 

2M 

kJ b x * ( t- 10 - 5 ( 0 , (t-0,+1, ±2, ...,) (A 1 .27) 


The boundary conditions for this series are 


b 

o 


- 1 , 


2M 

2 

k-0 


<00 


(A 1 .28) 
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and since the innovations are independent and the x(t)'s and the innova- 
tions are mutually independent, the expectations of the following products 
must be zero: 

E{x(s) C (t) } =0 for all s £ t - 1 (AI.29) 

e£?(s) I (t)} =0 for al I s ^ t (AI.30) 

The autocovari ance matrix elements for a stationary process are 

\ 

defined as: 

E (x(t - p) x (t- k) }=C(p- k) (A 1.31) 

\ 

The autocovariance matrix elements can be related to the expansion con- 
stants, b^, and to each other by the Y ule-W al ker eauati ons. These equa- 
tions are derived by multiplying both sides of Equation AI.27 by x(t- p) 
and taking the expectation: 

2M 

E (x(t - p) £ bkx(t- k)} — £ bkE ( x (t - p)x(t- k)} 
ic=0 k=0 

= E {x(t-p) i (t)}=0 (A1. 3 2) 

p=l , 2, •••, 2M 

Substituting into (AI.32) wi..th the definition of the autocovariance 
matrix elements gives: 

2M 2M 

T b C(p-k)-C(p) + £ bC(p-k)«0 (A 1 .33) 

k=0 k k«l k 

This gives the Y ule-W al ker equations (b Q * 1 ): 


2H 

£ b C(p - k) * -C (p) 
k=l k 


(A 1 .34) 
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Equation AI.32 is similiar to an orthogonality statement about the auto- 
covariance elements and the Y ule-W al ker equations express a particular 
autocovariance as a weighted sum of the others. 

The variance for the process is defined as the expectation of the 
product of x(t) and §(t): 

2M , 

E ( x ( t) § (t)} =2 bkC(k) =c (AI.35) 

k=0 

This equation is used to estimate the variance of the process from the 
autocovariances and the autoregressive coefficients. In maximum entropy 
spectral estimation, this variance is called the error power of the process 
and is minimized by a least-squares criterion. 

An autoregressive series can also be used as a linear predictor. The 
series can be extrapolated one unit ahead in time by using the following 
equation: 

2M 

x ( t+l ) =§(t + l) - Z b.x(t-H-k) (A 1.36) 

k=l K 

The best linear one step predictor (in a least-squares sense) is given by 
(Koopmans, 1974, p. 228): 

2M 

x(t)=- Z bkx(t + 1 - k) (A1.37) 

k=l 

The one-step prediction error is equal to the innovation variance: 

E{[x(t+l)-x(t)fr = E^(t+l)}=a 2 (A 1.38) 

The autoregressive process can be thought of as high order Markov 
process. That is, the state or value of the series at time t depends only 
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upon the previous states or values. Markov processes are important to the 
study of random variables, probability, and discrete-time random processes 
(Thomas, 1969, p. 114). 

Mixed auto regressive- moving average processes are modeled by a com- 
bination of finite autoregressions and finite moving averages (Koopmans, 
1974, p. 240): 

q r 

S d x(t- p) = S a^Ct-k) (A 1.39) 

p=o p k=0 K 

This type of model can be viewed as a linear filter with an output of 
x(t) in response to a white noise input, §(t) . The transfer function for 
the filter can be obtained from the z-transform as: 


fc v jwk 

H(w) = (A 1.40) 

Z d e- JWP 

p»0 P 

The autoregressive-moving average process can also be modeled by a high- 
order recursive filter. The output of the filter, y(t), is described by 
(Otnes and Enochson, 1972, p. 89): 
q r 

yet) = £ g x(t - p) + 2 - k) (A1.41) 

p»0 P k=l 

The filter is modeled by the parameters, gp and h^. The complex transfer 
function for this filter is: 


H(w) 


£ g e 

p*0 p 

i - £ 
k=l ^ 


-jwpAt. 


-jwkAt 


(A 1.42) 
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H. The Power Spectrum of an A utoregressi ve-M oving Average Process 

By using a filtering model, the power spectrum for a moving average 
process can be estimated by (w= 2nk£t) 


_2 2M ._„,k 

S (w) =-f Z a, e JCL 
x 2rr |<=0 k 


(A 1.43) 


where x(t) is modeled as the output of a digital filter with white noise 
input §(t) and a complex transfer function of: 


2M i- w k 

H (w) = E a. e-J w 
k=0 k 


(A 1 .44) 


The spectrum for a finite autoregressive process can be defined from 
the theory of inverting linear filters and z-transforms (Koopmans, 1974, 
p. 217): 


2 1 1 1 

S X (UJ) = 2 tt iHeWyi? * . 

. s be 
k=0 


(A 1.45) 


The spectrum for a autoregress ive-moving average process is given by: 


y W ) 


t 2 L,...J 2 . a 2 


r 

E 

v' J “ k 

k=<) 


q 

E 

d e-i w P 

p=o 

P 


(A 1.46) 
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I . W indow F unctions 

The concept of a window function is very useful in the study of spec- 
tral analysis and sampled data systems. The window function serves as a 
mathematical tool to "bridge the gap" between theoretical concepts and 
empirical analysis techniques. Its most important use is to model the 
effect of finite data and finite observation times on estimated autocor- 
relation functions and estimated power spectra. There are three types of 
window functions to consider; the data window, the lag window, and the 
spectral window. 

The data window (also called the observation window) models the effect 
of a finite observation time. A continuous time function, x(t), which may 
hypothetically exist for all time, can be observed for only a finite time, 
T^. This function may be periodic, stochastic, or mixed and its spectral 
bandwidth may be very large but all of its properties must be determined 
from those values in the observation interval, T^, which are known. From 
this finite observation, one usually tries to obtain esti mates of such 
characteristic parameters as average value, mean-square value, autocorrela- 
tion function, and power spectral density function. The length of obser- 
vation time (width of the data window) has a profound effect on this 
estimate. 

The inherent process of observation forces all data windows to be 
rectangular functions. The observed data function, denoted by a circumflex 
accent, is the product of the real function and the data window function: 

x(t) « h d (t)x(t) 


(A 1 . 47) 
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The estimated properties of x(t) will generally be different than the 
"true" properties of x(t). As a general rule, when the observation time 
becomes large, the. properties of x(t) become good estimates of the 
properties of x(t) and the effect of a finite observation time becomes 
minimized. This is especially true when the spectral bandwidth of x(t) is 
limited. The effect of finite observations on the amplitude spectral 
function can be seen by taking the Fourier transform of both sides of 
E quation A 1 .47. The resulting estimated amplitude spectrum, assuming the 
Fourier transform of x(t) exists, is obtained as: 


&u>) = 




Hd(W- \)x(\)d\ 


(A 1 .48) 


This convolution of the data window spectral function with the "true" 
amplitude spectrum causes a spectral smoothing effect and a loss in spec- 
tral resolution. Also, the spectral window will introduce spectral mixing, 
A rectangular window causes the most spectral mixing and to minimize this, 
seme type of window smoothing may be employed. A data function can be 
tapered at each end to minimize the discontinuity caused by the observa- 
tion. Various types of data windows and an analysis of their performance 
can be found in Otnes and Enochson (1972, p. 281). 

The I aa window is used primarily to model the effect of a finite 
autocovariance function, Co(t), where t is called the lag time or simply 
the lag. In the Blaciopati-Tukey method of spectral analysis, the lag window 
is also used to reduce the effect of statistical variance by emphasizing 
lag values close to zero lag. The apparent or observed autocovariance 
function, C (T) must be finite because the time function, x(t), is observed 
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only for a finite time. An unbiased estimate for the autocovarianc:e 
function has been defined as (Blackman and Tukey, 1958): 

+(V tr|)/2 

c o(") = vT f I J x(t ’ T/2)x(t + T/2)dt M <a * (AL49) 

-(TN - |t|)/2 

This definition produces an autocovariance function which is even and 
estimated over the lag interval -TN £t£+T n . For statistical reasons, this 
estimate is best for values of lag time near zero. The variance of the 
estimate becomes very large as the lag time approaches ±T^* Blackman and 
Tukey suggest using a lag window function to modify the shape of Co(t) so 
that lag times near zero are emphasized. 

The apparent or observed autocovariance function, when multiplied by 
a suitable weighting function (lag window function), is called the modified 
apparent autocovari ance function and its Fourier transform gives a smoothed 
estimate of the power spectrum of x(t). This modification produces an 
autocovariance estimator that is defined by the equation 

<$<t) = D(t)C o (t) (A I. SO) 

where the lag window function, D(t), may be chosen from a variety of' 
functions that are c oe nr o nly used for the purpose of spectral smoothing. 

Lag window functions have been very rigorously investigated for their 
properties in an effort to produce a window function which will nri 
spectral resolution and minimize spectral "leakage". Names for window 
functions cosmonly used in the literature include; Harming window, Hamming 
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window, Goodman window, Parzen window, Goodman-Enochson-Otues window, and 
Bartlett window. Generally speaking, all lag windows have a general shape 
like a cosine bell or a smoothed sine function. A rectangular window is 
commonly used in theoretical treatments of spectral analysis but usually 
not in practice because it has severe leakage problems. Further discussions 
of window functions can be found in Otnes and Enochson ( 1972 ), Koopmans 
( 1974 ), B lackman and Tukey ( 1958 ), and Papoulis ( 1973 ). 

The spectral window function models the spectral smoothing effect 
produced by a lag window. The spectral window function is defined as the 
Fourier transform of the lag window function. The expected value of a 
collection of estimated spectral density functions is the convolution of 
the spectral window function with the "true" spectral density of x(t). 

The various relationships between the lag and spectral windows and the 


"true" and estimated spectra are summarized below: 

P(f) (true spectrum) (A 1.51) 

p(f) - (a single spectral estimate) (AI.52) 

= ^{C o (T)D(' r )} (A 1 . 53 ) 

0(f) =^{D(T)} (spectral window) (AI.54) 

i+-» 

Etf(f) } = J 0 (f - X)P(X)d\ (spectral smoothing) (AI.55) 

_CD 

J. Calculus of V ariations 


The calculus of variations problem with constant Lagrangian multipliers 
and integral constraints is formulated as follows: 

1) Given a definite integral 
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b 

H = J F ( x , p ( x » d x (A 1 .56) 

a 

the problem is to maximize H with respect to the function p(x) where 
p(x) has N constraints as given by: 


& 


k(x,p(x»dx= ^ 


k= 1, 2, 3, 


N. 


(A 1 . 57) 


The integrals must all be finite and the limits of integration are 
over all defined values of the independent variable, x. F(x,p(x» and 
Qk(x,P(x» are explicit functions of x and p(x). p(x) is a function 
of x alone. 

2) Substitute for the function to be maximized (in this case p(x» 
the following modified function which has N arbitrary "perturbation" 
functi ons: 


P(x) = P(x) + e^r^x) + i^fx) + ••• (A 1.58) 

£(*)- P(x)+ J %\<x) (A 1.59) 

This produces, a function with p(x) and N additional degrees of freedom. 

3) Next, form the following auxiliary function 

N k 

w*H(e^, ®2» ••• "F £ “ ^3 (A 1 . 60) 

k=l a 

wbere Q k * Q k (p(x) , e^, ... e^) . 

4) w is minimized with respect to eack by evaluating the following 
derivatives at al I 0: 




v° 
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= 0 (A 1.61 ) 

all e. =0 
k 

This procedure will yield N equations that can be used to solve for 
N constraint constants. The partial derivatives can be expanded by 
noting that: 

3c k ' Sp* *k Se k " 3p BC k 
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*» 

*‘k 

all 


V x) 




m 


A 

9p 


c k =0 


all 



(A 1 .65) 


^=° 


(A 1 .66) 


The partial derivatives for ware reduced to: 




3? 


all 


e k =0 


N 

+ 2 

m=l 




m 


A- 

3? 


all 


*k =0 


]dx=0 


(A 1 . 67) 


all 


«k=° 


5) Since the integral of Equation is zero for any arbitrary "pertur- 

, only two possibilities exist. Either r^(x) =0 
which is the trivial solution or else: 


bation" function r^(x) 


JF 

dp(x) 


-ns* 


N <Q 

i x 

nr^/. 


ap(x) 


=o 


(A 1 . 68) 


all $ a 1 1 « k * 0 

Equation A 1.68 is called the E uler-L agrange equation and may be 
written in an alternate way as: 


l 0P(x,J(x))+S \Q (x,p(x))J=0 (A 1. 69) 

3p(x) all c = 0 

k 


The Euler-Lagrange equation will generally contain N Lagrange multi- 
pliers, the independent variable x, and the dependent vari abl e. p(x) . 
6) The Euler-Lagrange e<juation is solved for p(x) in terms of the 
\ m ‘S and z. This solution will generally be very complicated and 
quite often transcendental. The explicit solution for p(x) will be 
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represented by Vex, X^,X 2 > •••X^). This solution is then substituted 
into the N constraint equations 


& 


k(x,V(s,X 1> X 2 , ••• X N ))dx 


(A 1.70) 


and the N equations are solved for the N values of X , the Lagrangian 


m 


multi pi i ers. 

7) The solution is completed when these X m 's are used in the explicit 


equation for p(x) to give: 


p(x) — V(x,X^,\ 2 » ••• X^) 


(A 1 . 71) 


The calculus of variations problem with a continuous Lagrangian multi- 
plier function is formulated as follows: 

1) Given the definite integral 


H 




F(x,p(x))dx 


(A l.n) 


a 

the constraints are now assumed to form a continuous function and the 
constraining equation becomes 


b 

J* Q(T,x,p(x))dx*$(T) (A 1 . 73) 

a 

where the continuous parameter r replaces the index k. 

2) Substitute for the function to be 7.*d ? the following modified 

function which contains an arbitrary "perturbation" function n('l',x) and 
an c(t) continuous multiplier that is independent of x. The modified 
function becomes: 

p(x) * p(x) + f C(T)T|(T,x)dT 
all T 


( A 1 . 74) 
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Note that the interval of t for #(t) must be the same as that for 
e(r) and ,,(t,x) or there will not be enough lldegrees of freedom" in 
the product c(t)ti(t,x). 

3) Next, form the following auxiliary function using p(x): 


w = H (e, t) ±J X(t) [J Q(T,x,p)dx- 0(T)]dT (AI.75) 

al I t a 

4) Calculate the partial derivatives of w with respect to s(t) and 
evaluate this partial derivative at c(t)=0 to minimize the function 
w at the poi nt p(x) = p(x) . 

The partial derivatives are expanded as before: 


a? 

&«(t) 


<£_ 
5®( t) 


e(T) =0 


ap 

3p A “^< T > 


\ 

€(T) = 0 


C (T) • 0 


. 2 iL 

- g* a«(r) 


A 


e(T) = 0 


12 - 

3«(T) I 

«(T) =0 


ap 


ae(T) 


e(T) =0 


(A 1 . 76) 


(A 1.77) 


(A 1 .78) 


It is important to note that the partial derivative of p(x) with 
respect to c(t) can be reduced to a function of x alone as follows: 





C (T) *0 


= Itt^CpOO + J e(T)n(T,x)dT] 
all t 


e(T) =0 


all T 

= I T|(T,x)dT 
all t 



(T) =0 


(A 1 . 79) 
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The integral of ti(t,x) over t leaves a function of x alone, therefore: 


Sp(s) 

S«(t) ' 

g(t)=0 



= K(x) 


(A 1 .80) 


With these various partial derivative expansions, the partial deriva- 
tive of w becomes: 


2* 

3e(t) , 

e(t)=0 



I dx + 
®( t )=0 


J 

all 


A<t) 

t a 


m 

ae(T) 


dxdr 


(A 1.81) 


Combining the integrals and reducing gives: 


2a 

d«(*r) 


b 

• -J 

e(T)=0 a 


K(x) 


± I 
a11 T 


dr]dx = 0 


(A1.82) 


5) Since the integral is zero for every arbitrary "perturbation" 
function K (x), the only nontrivial solution is: 

:?(*)) 

9p(*) 

c(t)*0 c( t)=0 

Equation AI.83 is the E uler-L agrange equation for a continuous con- 
straint function defined by an integral constraint. This equation 
will contain a continuous Lagrangian multiplier, X(t), the independent 
variable x, and the dependent variable, p(x). 

6) The E uler-L agrange equation must be solved for p(x) in terms of 
the multiplier function X(-r) and x. In general the solution will in- 
volve a transcendental equation in p(x). An explicit solution would be: 


±J 


all t 


X(t) 


?0(r x,p(x)) 


A . 

9p(x) 


dy “ 0 


(AI.83) 


p(x) - V Ji (x,X(t)) 


(A 1 . 84) 
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7) This value of p(x) is then substituted into the constraint 
equation to give: 

£q( t,x,V x O:,X(T)))dx -0(f) (AI.8S) 

a 

This integral equation must be solved to yield X(-r) as some function 
of #(t) represented by: 

X(t) = R.(0(t)) (A 1. 86) 

0 

Again this solution may be transcendental. 

8) The solution for p(x) to maximize H is completed when the solution 
for X(t) is used in Equation AI.84 to give: 

p(x) = V x (x,R 0 ) (A 1.87) 
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XXII. APPENDIX II - FOURIER ANALYSIS 

A. Fourier Series - Equivalent M athematical Forms 

A continuous function f(x) is periodic in x if it is defined for all 
real x and if 

f(x+nT) = f(x) ( a 2 . 1 ) 

\ 

where T is called the period of f(x) and n is any positive or negative 
integer including zero. 

The familiar Fourier series representation of f(x) is given as: 

f(x) =a o + n | 1 (a n cos -Tx + b n Sl.n -T x) (A 2.2) 

The Fourier coefficients for (A 2.2) are computed by the following integrals: 

1 MT/2 

a ** - o f(x)dx (A 2.3) 

o T -772 

2 ctf772 2rm 

a =- o f(x) cosf-T x)dx (A2.4) 

n T -772 

2 <+772 2_ 

b =- O f(x) si n ( r^x)dx (A2.S) 

n T -772 


The complex representation of a Fourier series is obtained by using 
Euler's equation, e ia = cosa + i sina, to replace the sin and cos terms of 
(A 2.2). When this is done, the series can be written as: 


_ . . 2rm * , 2nn 

® + 2 ~ t” -J --x 

f (x) =k + S [k e 1 + k e T ] 
o n*l n n 


(A 2.6) 
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The complex coefficients are related to the a's and b's as follows: 

\k\ 


k = a n 
o o 


k = rr (a - \ b ) 
n 2 n 1 n 


n 1 2 n n 

-b_ 

0 n = Arc can ( r) 


(A 2.7) 


The complex series representation is most often seen in the following 
compact form 

+s> +j X 


See T 

.=-• n 

(A 2.8) 

■ < - 2 TTO 

+ T/2 j — x 

\ f(x)e dx 

Vrr/'l 

(A 2.9) 


n T -772 

■ , * 
with c = c • 

-n n 

Another common representation of the Fourier series is the phase angle 
representation. This representation is obtained from Equation A2. 2 by 
using the substitutions a R = d n cos$ n and b R =-d sin$ . After several 


trignometric manipulations we obtain: 


2 m 


f(x) = a + £. d cos (■=— x + ^ ) 
o n*l n v n y 


(A 2.1 0) 


The values for d n> and $ n are computed from the a's and b's as follows: 

d = (a 2 + b 2 )^ 
n n n 

(A2.ll) 

=Arctan(~) 

n 

The relationships between the complex representations and the phase 


angle representations are summarized below: 
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a) a = k +k 
n n n 


: n ' } <a n 


b =j(k -K) 

n n n 


k n "l (a n +i b n> 


b) a n = c n +c -n 


b n - j<c„- c. n ) 


c =l(a - j b ) 
n 7 n J n' 

c =l(a + j b ) 
-n 7 n ' n' 


|c | = $(*l +h b k 


-b 

0 = Arctan( — -) 

n a y 


c) a = d cos 4, 
' n n 


b n — d n sin «n 


2 2 b 

d = (a +b )* 
n n n 

-b 

0 = Arc tan ( ) 

n a 

n 


c 

n z n 


c =-; d e-J' 
-n 2 n 


b n = 21c n I 


lm{c } 

«n = Arct> ° [ s ^n ] 


(A 2 . 1 2 ) 


(A 2.13) 


(A 2.14) 


(A 2. IS) 


(A 2.16) 


Each representation of the Fourier series has its useful applications. 
Quite often the phase angle representation is used in discussions of net- 
work theory and wave propagation. For data analysis, the compact complex 
representation is used because it is identical to the digital Fourier trans- 
form and also because it is notationally convenient. 


B. Fourier Series - D iscrete I ndependent V ariable 

An infinite periodic time series can be represented by a digital 
Fourier series. Using the compact complex representation and assuming 
equally spaced x-values we get: 
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+M 

f(kAx) = n=5l 


c e 
n 


+j ^T^kAx 


(A 2.17) 


There are only (2M + 1) values of c n because for a finite number of values 
of f(x) in the Fourier period, T, only a finite number of complex ampli- 
tudes can be uniquely specified. This result points out the important fact 
that a digital Fourier series must, of necessity, be spectrally limited. 

The complex amplitudes can be exactly specified by the following digital 
routine which is obtained by using a Riemann sum approximation for the 
integral of (A2.9): 

ax +M -j-^nkAx 

C n T k=^ f(kAx)e (A 2.18) 

Equations A2.I7 and A2.IS form a digital Fourier transform pair if the in- 
crement Ax is such that Ax =77(2M + 1) • The c^ 's calculated from (A 2.1 S) 
can then be used in (A2.17) to reproduce the exact f(kAx) -values. 


C. Fouri er Transform Pair 


The Fourier series can be used to derive the Fourier integral and 
Fourier transform. If a Fourier integral is to exist for an arbitrary 
function f(x) (not necessarily periodic) the function must be piecewise 
continuous on every finite interval (Riemann i ntegrabi I ity) , it must have 
a finite average value, and it must be absolutely integrable (sufficient 
but not necessary). 

The Fourier integral may be written as 

f(x) ^A(w) cos vx + B(w) sinwx]dw (A2.I9) 



422 


w here: 


4cd 


A(w) -,r f(u) cos wu du 


4cd 


B(w) * J* f(u) sinwu du 


(A 2.20) 
(A 2.21 ) 


The Fourier transform is the complex representation of the Fourier integral, 
Using Euler's equation, the Fourier integral becomes 

1 r - * 0 jvx**® +TVU +iwx|+= 

f ( x ) ,1 [e- J f(u)e ^du +e J f (u)e-J wudu]dw (A 2 .22) 


0 


which can be reduced to: 

+», n +°° 


f(x) m k 1 e+jVX 0 f(u)e ’J wu 


du]dw 


(A 2.23) 


Equation A2.23 is used to define the F ouri er transform pair: 

+». 


f(x) I e+ivXF ( v > dw 


F(w) f 4 f(x)e‘ :,wx , 


dx 


(A2.24) 


(A 2.25) 


Many functions that appear in physics and mathematics are formulated 
in terms of the Fourier transform and thus are associated with Fourier 
transform pairs. Tvo ccrtnnon examples are: 1) position and momentum in 

quantum mechanics and 2) autocorrelation and power spectral density in 
signal processing. The autocorrelation function, R(T) of a random process 
and the power spectral density function, S(w), of that process form a 
Fourier transform pair: 


Sew) «J R(T)e" JtOT dT 


(A 2.26) 



An alternate mathematical form that is often seen is obtained by substi- 
tuti ng 2tTf = w to give: 

P(f) =v R( T ) e ’ j2rrfT dT (A 2.28) 

-CD 

R(T) =J* P(f)e +j2rrfT df (A 2.29) 

-CO 

D. Digital F ourier Transform 

The digital (discrete) Fourier transform is obtained by using a 
Riemann sum appro xima tion for the integral of Equation (A 2.24): 

f(kAx) = -L^w E F(n£v)e +jnAwkAx (A2.30) 

2TT n--N 

The spectral amplitudes are obtained" frctu the sample values of f(k&x) by 
the Riemann sum: 

F(n4w)=Ax E°f(kAx)e' jnAvkAx (A2.3I) 

k*-» 

Other mathematical forms of these equations are often used to develop the 
theory and practical application of digital Fourier transforms. 

E. Special Functions 

There are several special functions which occur repeatedly in the 
mathematics associated with Fourier analysis. Three of the most important 
are the Kronecker delta, the Dirac delta and the sine function. The fol- 
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loving tabulations of the various forms of these three functions are in- 
cluded because they have proved to be valuable for use in a variety of 
deri vati ons. 

The Kronecker delta plays an important part in Einstein simulation 
notation and in the formulation of digital Fourier analysis. The Kronecker 
delta is defined as follows: 

™f k =n k,n are integers 


kn 


1 4 


k ^ n 


(A 2.32) 


Some of the various mathematical forms for the Kronecker delta function 


are: 


6 kn 

si nn(k - n) 
n(k - n) 

(A 2.33) 


li +T/2 +j ~<k-n)x 


6 kn- 

e dx 

(A 2.3.4) 

T -T 12 


5 kn s 

~J" HT e+j 9 {k-n)d9 <T«2tt) 

(A 2.35) 

IT 



0 +T / 2 ^ ? 

- J* cos( I 2 kt) cos (zr nt)dt 

I J T/2 T T 

(A 2.36) 


■Hi 1 +i (2M + 1) (k-n)p 

6 kn*p»? M 2M +1 6 (A 2.37) 


This last form was discovered in the derivation of the spectral mixing for- 
mula and can also be written alternately as: 


M 


SIT 


" ( 2 K +1 ) [ I +2 2 cos g t ^ -n)p] 
0*1 


(A 2.38) 
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The validity of this equation was shown using a double precision computer 
program. The magnitude of the indexes was shown to be restricted to the 
ranges; Osk£M, OSniM. 

The Dirac delta function (or impulse function) plays the same type of 
mathematical role in continuous functions as the Kronecker delta does in 
discrete functions. The Dixae delta is defined as follows: 


x “ y 


6 (x - y) = { ; 

otherwise 

(A 2.39) 

o> 

X s 

Q_ 

X 

II 

i— 1 

(A 2 *40) 

al 1 x 


J f (x)6 (x - y)dx * fey) 

(A 2.41 ) 

al 1 x 


Equation A2.41 is the sifting (or sampling) property of the Dirac delta. 

The Dirac delta function also has the following integral 

defi niti on 

(Cauchy principle value): 


§(x-y). 

(A2.42) 

-CO 


The Dirac delta can also be represented in the following 

mathematical forms 

S(V-v)= lim T sinnT(x-V) 

T-*oo ttT(x-y) 

(A 2.43) 

2 2 

6(x - y ) ■ lim T e" 111 (x-y) 

T-*0O 

(A 2.44) 

e ( x ±v ) • lim lsin(x±v)T 
T-» n (x±y)T 

(A 2.45) 
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For conceptual purposes, the Dirac delta is often represented by a rectan- 
gle function of height T, width NT, and unity area: 


6 (x - y) • lim T rect(T(x-y)} 
T -*« 


(A2.460 


The sine function appears in most discussions of Fourier analysis and 
sampling theory. It is the Fourier transform of a rectangle function and 
can be used to approximate a Dirac delta function. The normalized sine 
function i s defined as 


sine(x) = £i , ilI2L 

TTX 


(A2.47) 


and has the following integral definitions: 


-fco 


sin TTX 
TTX 


So 


+® 

si nmx 


dx . i 


r Tt/ 2 

dx- 1 0 

m>0 

maO 

-tt/2 

m<0 


(A 2.48) 


A more generalized form of the sine function can be written as: 


AT sinc(r-) 



(A 2.49) 


Figure A2-1 show a plot of the generalized sine function. The peak ampli- 
tude of AT m occurs at w - w. The zeros of the sine function occur at 

w - W 0 + a ijr“, n- 1, 2, 3 ••• and the bandwidth between first nulls is 
m 

BWFN - 2rr/T m - 



“c 


Figure A2-I. A Generalized Sine Function 


The sine function has the following useful mathematical properties 


1 +* sinlw-WoJT^ A 

2TT J AT m [ (W- W )T =2 

— -» o — m 4 


lim AT [• 
in 


sin({»- (D )T 
o m 

(0U- UUjT 
u in 


] = TTA 6 (id- UL) ) 
o 


(A I 


(A2 
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XXIII. APPENDIX III - COMPUTER ALGORITHMS 

A. program 01 - Discrete Fourier Transform Testing 429 

B. program 02 - Estimated Kronecker Delta Function 435 

C. program 03 - Estimated Spectral Amplitudes using 

Kronecker Delta 439 

D. program 04 - D iscrete Fourier Transform A nalysis 441 

E. program 05 - Fast Fourier Transform Spectral Density 453 

F. program 06 - Estimation of Si ngle-Si newave Parameters 460 

G. program 07 - Maximum E ntropy Spectral Analysis 466 
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4 JOS 'RUSSH_L * . TI«t=^O.PACLS=?£ 

c 

C FPCGPAV 01 


C 

C 

C 

C 

C 

C 

c 

1 

2 

3 

4 5 
C 
C 
C 

5 

6 

C 

7 

£ 

Q 

10 
1 1 
12 

C 

1 3 
1 4 
15 

C 

c 


DISCRETE F CUP I £ P TRANSFORM TESTING 

THIS PROGRAM IS D SL C TO ANALYZE THE. EfFECT OF 
VARIOUS SAMPLING CONCITICNS CN ThF DISCRETE FOUR 1 £ P 
TRANSFORM ESTIMATE CF ThF SPECTRUM CF A TIME SERIES. 
THE INPUT T IMfc SERIES IS GENERATED IN THE PRCGR AM. 

DIMENSION F VALUE ( 2C 1 ) , Fx ( 8 04 ) , A (40 0 ) »t> <4 00 ) 

CO b 1 = 1.400 
A (I )=C .0 
r i ( i ) =o.o 

input DATA values 


A C 1 ) =4 . 0 
B( 3) =1 • 0 


TF =1.0 
7N=3. 1 
NS=2 1 
FNS=NS 

toel=tn/fn s 

SC Al 0 = 2 . 0 

N LS=N S 

N M A X = (NS-1 )/2 
DC V= 0. 0 

GENERATE CATA VALU C S. EQLA T I ON C 1 20 . 1 ) 


16 CALL FSRS(FX ,A ,P. ,CCV ,M$ ,TDEL ,TP ) 

17 FPtNT 9 

le 9 FCRMAT( * i • ,1 I X ,'K ‘ *7X , « T IME» , 13X , • F< TIME ) •/ ) 

19 DO 1 0 I =1, NS 

20 FVAL'JE (I '=F X ( I ) 

21 II=I-NMAX-1 

22 XDEL f A =1 IATCEL 

23 10 PP INT. I I , X CELT A, FX ( I , 

C 

C COMMUTE COMPLEX FOURIER AMPLITUDES. E GUAT I 0N< 1 2D • 6 ) 


24 CALL LNSPTM(F VALLE ,A ,fc .MS.CCV) 

25 PRINT 34 

26 34 FORM 4T ( M • ,4X , »N • . 3X , • CC-VALUE • , 4X, * CM AG • . EX 

1 , 'CPHASE ’ * 6X. * F P£ C -H 2 * ,7X,'CMAC-C8»/) 
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27" 

28 

25 

30 

31 

32 

33 

34 
3 ‘3 

36 

37 

38 

39 

40 

41 

42 
4 3 
4 4 

45 

46 
4 7 


4 4 
45 

50 

51 

52 

5 3 

54 

55 

56 

57 

58 
55 
60 


6 1 

62 

63 

64 

65 


NL v AX = (NLS- 1 ) / c 

00 35 1=1, KLMAX 

X i = I 

CMAG= ( (MI ) * A ( I ) +e ( I >*b( I) >>*C.5 )/2, 

C,v=c V4G/SCALE 
CMAGCb=20.*AL0GlC(CM) 

A7cST = 5 ( I > / A £ 1 ) 

A 75 ST = ATE S.* AT6S i 
ATFST = ATEST* * 0 , 5 
ATMOOO.-ATOST 
IF< AT ) -20 • 20 , 2 i 

20 CPHA3E = • 38333836 
GG TO 22 

21 CONTINUE 

6 A =— 8 (I ) / A (I ) 

CPHASE = AT AM BA ) 

C PH A SE = C PH AS £ 1 57 *29577 
FNLS=NLS 

22 FPE0=X I / ( T CEL TFNLS ) 

35 °Hl NT 36.1 ,DC V ,e MAG .CPHASE ,F PE C, CM AGCH 

36 FuRMATC 16, 4E 12,3 ,F 1 0 * 2 > 

C 

C COMPUTE The ESTIvaTEC T I ML SERIES, E CG AT I ON ( 1 20 . 6 ) 

C 

N 3 =4 * N S - 3 
TOLL = T OE L / 4 
NMAX= ( NS- l )/2 
TF = TN 

CALL F5PS(FX,A,B ,0CV,NS.TCEL,TF1 
PAINT 90 

90 FCPMAT ( e l ' ,11X,*K',7X, 'TIME' , 1IX 
1 , * E S T 1 MATE 0 F ( TI ME ) ' /) 

OC 100 1=1 , N S 

1 I =1 -NVAX-1 

XO EL T A =1 I * TOEL 
100 Pft INT , I I , XCELTA, FX ( I ) 

501 STOP 
t.MO 

C LINE SPECTRUM SUBROUTINE 

c 

C THIS SUBROUTINE GENERATES THE FCUPIEF C Cc.FF I c I E NTS 
C A ANC 6 F CP NLS SAMPLES OF THE FUNCTION, FvALUt. 

C 


SUSP 0 LT I Nt LNSPTM( FVALLt » A * b ,NLS,DCV) 
DIMENSION FVALUt (201 ) , A ( 400 ) , t C 40 0 ) 
NL*AX=( NLS-1 ) / 2 
PI=3. 14 15526 
XNN=NLS 
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66 

67 

68 

69 

70 

71 

72 

73 

74 

75 

76 

77 

78 
7<? 
SO 
6 1 
82 
S3 
64 
8 6 
66 
87 


es 

8 *5 
CO 

5 1 

92 

93 

94 

OS 

96 

97 

98 

;9 

100 

101 

102 

103 


or v=o. 

00 15 J =1. NLS 

15 CC V=OCV+FVALU£ ( J ) 

DC V=DC V/XNN 

DC 15 J ■= 1 * SL S 

16 FvAL'JE< J)=FVALU£ IJ >-CCV 
D) 10 N = 1 » NL VAX 

ASUM = 0 • 

6 S UM = 0 • 

XN=N 

CC 5 J =1 .N LS 
J J=J-NL*AX-1 

XJJ =JJ 

ACG=2tO*PI *XNAXJ J/XNN 
ASUM=CCS(ASG)*FVAIUE(J)+ASU* 

5 8 SUM -5 IN ( APG ) *F VALUE ( J ) + USUW 

A (N) =2 ,0*A£UM/XNN 
10 8( N) =2.0*2 SLM/XNN 

00 17 J —3. • NLS 

17 FVALUe ( J) =FVALU8 ( J )+CCV 

TUP N 
CNC 

C F CUP IdF DEFIES 3U6PCUTIME 

C 

C THIS SU6R0UT INc GENERATES SAMFLES OF THE CATA 
C FUNCTION# F X( I ) * USING THE FCUSIES (C E F F I C I E N T S . 

C A ( I ) . 8(1). AND OC-VALUE, DCV, AND THE NUMBER OF 

C DESIftfiD SAMFLES. NS. TDEL IS THE SAMPLING PERIOD 
C AND TF IS THE FCUSIES FEPiCC. 

C 

SUBROUTINE FSSSIFX »A#EtCCV*NS»TDEL«TF> 

DIMENSION F X C e04) .A {400). 8(400) 

NMAX = { NS- 1 )/ 2 
PI =3.1415926 
XN3=N5 

DC 10 1=1, N5 
K= I -NMAX- 1 
XK -K 
<=XX=0 . 

DO 5 N= 1 .NMAX 
XN = N 

ASG=(2.0AFIAXN«XK*TCfcl )/TF 
5 F XK= F XK + A ( N)*CUS<ARG)*B(N)*SIN(ASG) 

10 FX( I ) = FXK + DCV 
RE TUP N 
END 


3£ NT S Y 
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TINE 


F < 7 1 M t > 


-10 

-0.1476 1 >03 

01 

-C. 436922 CE 

Cl 

-9 

-0.1 328570 E 

01 

-0 . 1S05765E 

Cl 

-e 

-0 . 1 1 809526. 

01 

0 . 19474605 

Oj. 

-7 

-0 .10333J3C 

01 

o • 3 22452 IF 

Cl 

-6 

-0.6657141 6 

00 

0 .38466335 

01 

-5 

-0. 73609506 

CO 

-C. 1 273664E 

Cl 

-4 

-0 .5 3047606 

CO 

-C .22799 276 

Cl 

-3 

-0.44265705 

00 

— 0 .4625533c 

01 

-2 

-0. 295238C6 

CO 

- C. 46276 02t 

OC 

-1 

-O.14761Q0E 

00 

0 .20479 1 IE 

01 

0 

C. OOCOOOOE 

00 

0.40000006 

01 

1 

0.1476190F 

00 

0. 275066 SF 

01 

2 

0 .2952380F 

00 

-0 . 17796445 

01 

3 

0.442657 OF 

CO 

-C. 26643365 

Cl 

4 

0 .5>v047o0t 

00 

- C .4 26 16391 

Cl 

5 

0.73609506 

00 

0.57596775 

CO 

6 

0. 6657141c 

00 

0.2 1 777 1 3f 

01 

7 

0 .1 033333 E 

01 

0 .45003665 

01 

e 

0. 1 1 £09525 

01 

0 ,1415427r 

01 

9 

0.1 3285 70 E 

01 

-C. 19 6 51 1 9E' 

Cl 

10 

0. 1476190E 

01 

-0.35 214186 

01 


DC — VALUE. 

CN4G 

CFF4SE 


FFEQ-FZ 


-0.132E:. 

00 

0 . 1466 UO 

- 0.41 4F 

CC 

C.323E 

00 

-O .132E 

00 

0.272E : 00 

-0 .5936 

00 

0 .6456 

CO 

-0.132E 

00 

0 . 2 00 i : . 01 . 

-0 . 1 C 96 

00 

C .9686 

00 

-0.132E 

UO 

0 . 136F CC 

0 .1 £46 

01 

0. 1 29 E 

Cl 

-0. 1326 

00 

0.741 6-01 

0.7166 

01 

0.161E 

01 

-0.1 325 

00 

0. 4276- Ul 

0.2C7E 

02 

0.1 94 F 

01 

- 0 . 1 326 

00 

0 .3636-0 1 

0 .4 6 3 E 

02 

0 . 2 2 6E 

Cl 

-O .132E 

CO 

0.647E-0I 

0.771E 

02 

0 .2686 

01 

-0.132E 

00 

O.280E OC 

0.8666 

C 2 

C.29CE 

Cl 

-O .132E 

00 

0.2bl E 00 

-0 «S95E 

02 

0 .323E 

01 


C MA G— 08 

-22.71 
-19. Oe 
0.02 
-20. 6 5 
-2b* o2 
-33.42 
-34. £2 
-29 .61 
-14.42 
-17.69 
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K 

T 1 


£ ST I fc-ATEC 

F (T 1 V E ) 

-40 

-0 .1476190t 

v i 

-0 .4389 17 IF 

01 

-39 

-G. 1439285E 

01 

-0.417278 H . 

01 

-38 

-0. 14023808 

0 1 

-C . 366666 95; 

Cl 

-37 

-C.1365476E. 

ut 

-0 .2P58696E 

01 

-36 ■ 

-0. 1 3285708 

Cl 

-C. 16C5771E 

01 

-35 

-0 . 1 29 2066E 

0 1 

— C.640363 5E 

CC 

-34 

-0.1254761E 

01 

0 .462859 OE 

00 

-33 

-0.121 78S6E 

Cl 

0. 1348933F 

Cl 

-32 

-0. 1180952 E 

0 1 

0 • 19474708 

C 1 

-31 

-0. 1 14 4 04 75 

01 

0. 23 0489 3F 

01 

-30 

-0.1107142£ 

01 

C. 256098 66 

C 1 

-29 

-0 . 1070237 F 

Ul 

0 . 28735995 

01 

- 2e 

-0.1033333E : 

01 

0. 3324809E 

01 

-27 

-0 .996428 3 F 

00 

C* 2 8568748 

Cl 

-26 

-0 « 9 59 5236 5 

00 

0.4276 63 9 E 

01 

-25 

-0. 92261S8E 

00 

0.4334717E 

01 

-24 

-0.S857141E 

00 

C.3846798C 

01 

-23 

-0. S4 8 8 09„3£ 

00 

0 *27982798 

01 

-22 

-0.61 19045E. 

00 

0. 137673 IE 

Cl 

-21 

-0 .7749998 5 

DO 

-0.910770 2E- 

-0 1 

-20 

-0,73809506. 

oc 

-0. 12738846 

Cl 

-19 

-0 .70 1 1903E 

00 

- 0. 197 9665E 

Cl 

-1 8 

-0 • 6642855 F 

00 

-0 .2236793E 

01 

-17 

-0.6273EC8E . 

00 

-0. 22705O9E 

01 

-16 

-0 .59 04760 E 

00 

— 0 • 2379 9 1 CE 

01 

-15 

- 0 • 55357 13 E 

00 

-0.2774965E 

01 

-14 

-0 .5 166665E 

00 

-C • 34550298 

01 

-13 

-O. 4~* 976 1 B E 

00 

-0 .4190885E 

01 

-12 

-0.4428570f . 

00 

-0.4625520£ 

01 

- 1 1 

-0.405S523£ 

00 

— C» 44466C9F 

Cl 

-10 

-0.3590475E 

00 

-0.35472 I6t 

01 

-c; 

-0. 232 1428E 

CO 

-C. 2093671E 

01 

—6 

-0.29523S0E 

00 

-C. 4 6 3 7 3 4 9E 

oc 

-7 

-0 • 258 3333 6 

oc 

0.918041 It 

00 

-6 

-0.2214285E 

CO 

0.176S814E 

01 

-5 

-0.1845238 E 

00 

0 .2067684E 

01 

- 4 

-0. 14 761 SOE 

00 

0.20476956 

01 

-3 

-0. 110714. 3E : 

00 

C. 2C642C3E 

Cl 

-2 

-0.7380950E - 

Ol 

0 . 2400929c 

01 

-1 

-0.36S0475E - 

Cl 

0.3116105E 

01 

0 

O.OI)OOOOOE 

00 

0*39999746 

Cl 

1 

0 • 36904 75 E- 

■01 

0 » 4669599E 

01 

2 

0.7380S50E- 

Cl 

C • 4 7 536 75E 

01 

3 

O.11071A3E 

00 

0 .4076528E 

01 
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4 

0.14761 >Ct 

00 

C. 27506566 

01 

5 

0. 1 8 4 62 38 3 

OC 

0.1I32101E 

OJ 

6 

0.221 4285 £ 

00 

-0. 33920938 

00 

7 

0.25833336 

00 

-0. 1 3357405 

01 

e 

0 .29522805 

OC 

- C . 17 796246 

01 

9 

0 .3321^28 8 

00 

-0 . 185558 IE 

01 

10 

0. 5690475E 

cc 

- 0 . 1896126c 

01 

11 

0 .4059523E 

00 

-0 * 2 1963856 

Cl 

12 

0 .44285706 

00 

-0.26643248 

01 

l 3 

f. *47976 18 6: 

00 

- C. 37424866 

01 

14 

0 . 5166665E 

OC 

-0 .44979528 

Cl 

1 5 

0.55357 136 

00 

-0 .47736546 

01 

1 6 

0 .59G47o0L 

OC 

- 0 • 436 18 146 

Cl 

17 

o ,627380e E 

00 

-0.330407 18 

Cl 

1 S 

0. 6642855E 

cc 

-0.187120PE 

01 

19 

0.7011903E 

OC 

-0.43938176 

au 

20 

C.7380<;50E 

00 

0.67595746 

00 

2.1 

0.7749998E 

00 

C. 13470738 

Cl 

22 

0 .8 1 1 9045E 

CO 

0 . 16 64 2 7 76 

01 

23 

0.6488093E 

00 

0. 1861 1638 

01 

24 

0 .88571418 

CO 

C* 2 177 ?Cer: 

01 

25 

o .9226188E 

00 

0 .27508446 

01 

26 

0. 95952366 

00 

o . 34 54 32 IE:. 

01 

27 

0 .9964 283E 

00 

c. 4 13 1 68 2E 

Cl 

28 

0 .1 0353336 

01 

0*45003476 

01 

217 

0. 10702378 

01 

C. 43722991 

01 

30 

0 • 1 1071 426 

0 1 

C. 37 14 0626 

Cl 

31 

0.1144047E 

01 

0.26530528 

01 

32 

0.1180C ;S2E 

01 

0.141 5442F 

01 

33 

0.121 78566 

0 1 

0 .23 160885 

OC 

34 

0.1254761 E 

01 

-0. 74356628 

00 

35 

0.12916668 

01 

- C. 146654 CE 

01 

36 

0 .1 3265706 

01 

-0 . 19851066 

01 

37 

0. 1 3654768 

Cl 

-0. 23894328 

Cl 

38 

0.14023S0E 

01 

-c. 2 7E995C6. 

Cl 

39 

0. 14392656 

01 

-0.3137555E 

01 

40 

0.14761906 

01 

-C. 3523 3748 

.01 


SST3P 
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i jQa 'RUSSELL 3 • . T I Me = 1 0 , PaC- 6S = 50 
C 

C Pr. O Gr. A M 32 


v_ 

ESTIMATED NFONcCKEF L'LLTA function 

r 



c 

THIS k.’JUTIMl' EVALUATES TH. APFROXIMATI JN Tl T HE 



KPGNECKcP. DeLTA FUNCTION. DOUBLE PRECISION IS USED 


c 

Te Ac SOLVE THE EXALT VALUES OF THc Urr -DIAGONAL 


c 

r 

elements * 

1 

L 

REALMS TH^TA i A M A T i C iLTCA*C'NN*OG*i k*TF *TN*DK*PI»DCU3 

2 


INT cuEP Ji h 


c 

INPUT DATA 

3 


N = S 

4 


M= 5 

5 


T N =1 . 

0 


TF =1 . 

7 


NN=2 *N + 1 

c 


DNN=NN 

■I 


MM=2*M+ 1 

a c 


PI=3«1415926535857 

1 1 

V- 

PRINT 6 5 

12 

d5 

FORMAT ( ' 1 ' ,5X . ■ Q'.7X»'i- ' *7X, »Oi_l.TQP • • 16X , ‘DELTOF •/ } 

13 


DC 80 1 = 1. NN 

14 


U=I-N-1 

c. 

1 - 


DU=Q 

1 a 


Cu 80 L-l i MM 

1 7 


R=L-M-1 

1 e 


0R=R 

1 -» 


AMAT =0 • 0 

2C 


DC 50 K= 1 . N 

21 


CK. = K 

22 


THl.TA = 2.*PI*OK*TN» ( DQ/TN-DF/TF )/0NN 

23 

60 

AMAT=DCGS( THE TA ) +AMAT 

24 


OcLTUR= (1 • +2 • *AMAT )/DNN 

2 5 

60 

PRINT 8 1 »Q *A «DELTQP. »C_ L TJR 

2 6 

61 

FORMAT ( Id, 16 ,F 10 .5. D 26# 16) 

27 


STOP 

26 


END 


GENTRY 



436 


Q R DELTCR 


-5 

-5 

1.00000 

-5 

-4 

0.00000 

-5 

-3 

- 0.00000 

-5 

-2 

0.00000 

-5 

-1 

- 0.00000 

-5 

0 

0.00000 

-5 

1 

- 0.00000 

-5 

2 

0.00000 

-5 

3 

- 0.00000 

-5 

4 

0.00000 

-5 

5 

- 0.00000 

-4 

-5 

0 .occco 

-4 

-4 

1.00000 

-4 

-3 

0.00000 

— 4 

-2 

- 0.00000 

-4 

-1 

0.00000 

-4 

0 

- O . OOOOO ' 

-4 

1 

0.00000 

-4 

2 

- 0.00000 

-4 

3 

0.00000 

-4 

4 

- 0.00000 

-4 

5 

0.00000 

-3 

-5 

- 0.00000 

-3 

-4 

0.00000 

-3 

-3 

1.00000 

-3 

-2 

0.00000 

-3 

-1 

- 0.00000 

-3 

0 

0.00000 

-3 

1 

- 0.00000 

-3 

2 

0.00000 

-3 

3 

- 0.00000 

-3 

4 

0.00000 

-3 

5 

- 0.00000 

-2 

-5 

0.00000 

-2 

-4 

- 0.00000 

-2 

-3 

0.00000 

-2 

-2 

1.00000 

-2 

-1 

0.00000 

-2 

0 

- 0.00000 

-2 

1 

0.00000 

-2 

2 

- 0.00000 

-2 

3 

0.00000 

-2 

4 

- 0.00000 

-2 

5 

0.00000 

-1 

-5 

- 0.00000 

-1 

-4 

0.00000 

-1 

-3 

- 0.00000 


DELTaR 

0.10000000000000000 01 
0 . 30246007718594890-13 
- 0 . 31389032787129430-13 
0 . 33723024372989130-13 
- 0 . 37404422993280280-13 
0 . 43215431233534220-13 
- 0 . 5189787993293232 D - 13 
0 . 66156676096924960-13 
- 0 . 90271224838012720-13 
0 . 14304214378636390-12 
— 0 « 30301014222 996 77 0—12 
0 . 3024600771859489 D - 13 
0.10000000000000000 01 
0 . 30246007718594890-13 
- 0 . 31389032787129430-13 
0.33 723024372 989130-13 
- O . 37404422993280280-13 
0 . 43215431233534220- 13 
- 0 . 5 18 S 7879932932320- 13 
0 . 66156676096924960-13 
- 0 . 90271224838612720-13 
0 . 14304214378636390-12 
- 0 . 31389032787129430-13 
0 . 30 24 60 0771 8594 89 D - 13 
0.10000000000000000 01 
0 . 30246007718594890-13 
- 0 . 31369032787129430-13 
0 . 33722024372989130- 13 
- 0 . 37404422993280280-13 
0.432154312 33534220-13 
- 0 . 51897879932932320-13 
0 . 66156676096924960-13 
- 0 . 90271224838612720-13 
0 . 3372302437298913 C -13 
- 0 . 31389032787129430-13 
0 . 30246007718594890- 13 
0.10000000000000000 01 
0 . 30 2460077 18594 690— 1 3 
- 0 . 31389032787129430-13 
0 . 33723024372989130-13 
- 0 . 37404422993280280- 13 
0 . 43215431233534220-13 
- O . 51 S 97879932932320-13 
0 . 66156076096924960-13 
- 0 . 37404422993280280-13 
0 . 33723024372989130-13 
- 0 . 31389032787129430-13 
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-1 

-1 

- I 

-1 

-1 

-1 

-1 

- I 

0 

0 


0 

0 

a 

a 

0 

1 
l 
l 
l 
i 
l 
l 
l 

1 
! 

1 

2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
3 
3 
3 
3 
3 
3 
3 
3 


-2 

0.00000 

-1 

1.00000 

a 

0.00000 

i 

-0.00000 

2 

0.00000 

3 

-0.00000 

4 

0.00000 

5 

-0.00000 

-5 

0.00000 

-4 

-0.00000 

-3 

0.00000 

-2 

-0.00000 

-1 

0.00000 

0 

1.00000 

1 

0.00000 

2 

-0.00000 

3 

0.00000 

4 

-0.00000 

5 

0.00000 

-5 

-0.00000 

-4 

0.00000 

-3 

-0.00000 

-2 

0.00000 

-1 

-0. 00000 

a 

0.00000 

i 

1.00000 

2 

0.00000 

■■> 

-0.00000 

4 

0.00000 

5 

-0.00000 

-5 

0.00000 

-4 

-0*00000 

-3 

0.00000 

-2 

-0.00000 

-1 

0.00000 

0 

-0.00000 

1 

0.00000 

2 

1.00000 

3 

0.00000 

4 

-0.00000 

5 

0.00000 

-5 

-0.00000 

-4 

0.00000 

-3 

-0.00000 

-2 

0.00000 

-1 

-0.00000 

0 

0.00000 

1 

-0.00000 

2 

0.00000 


0. 3024600771 85 94 89 D- 13 
0.10000000000000000 01 
0 . 30 24 60 077 1 859489 D- 13 
-0. 31 389032787129430-13 
0. 33723024372989130- 13 
-0. 37404422993280260-13 
0.43215431233534220- 13 
-0.51897679932932320-13 
0. 43215431 233534220- 13 
-0 . 37404422993280260- 13 
0. 33723024372989130-13 
-0.31389032 787129430-13 
0.30246007718594890-13 
O.IOOOOOOOOOOOOOCOO 01 
0.30246007718594890-13 
-0. 31 389032787129430-13 
0.33723024372989130-13 
-0 .37404422993280280— 13 
0. 432 15431 233534220-13 
-0.51897879932932320-13 
0.43215431233534220-13 
-0. 37404422993280280- 13 
0.33723024372989130-13 
-0. 31389032787129430-13 
0 .302460077 18594890- 13 
0.10000000000000000 01 
0. 302460077 1 8594890-13 
-0.31389032 78712 9430-13 
0.33723024372989130-13 
-0. 37404422993280280-13 
0.66156676096924960-13 
-0. 51 897879932932320-13 
0 .43215431233534220- 13 
-0.37404422993280280-13 
0. 33723024372989 130-13 
.-0.31389032 787129430-13 
0.30246007718594890-13 
0.10000000000000000 01 
0.30246007718594890-13 
-0. 3 1 3890327 87 129 4 3C- 13 
0.33723024372989130-13 
-0.90271224838612720-13 
0. 66156676096924960-13 
-0.51897879932932320-13 
0. 4321 5431 2 3353422 C- 13 
-0. 374044 229932 80 280- 13 
0.33723024372989130-13 
-0. 31 38 9032 7871294 30-13 
0.30 24600 7718594 890- 13 
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3 

3 

1.00000 

3 

4 

0.00000 

3 

S 

- 0.00000 

4 

-s 

0.00000 

4 

-4 

- 0.00000 

4 

-3 

0. 00000 

4 

-2 

- 0.00000 

4 

-1 

0.00000 

4 

0 

- 0.00000 

4 

1 

0.00000 

4 

2 

- 0.00000 

4 

3 

0.00000 

4 

4 

1.00000 

4 

S 

0.00000 

5 

-S 

- 0.00000 

5 

-4 

0.00000 

5 

-3 

- 0.00000 

5 

-2 

0.00000 

5 

-1 

- 0.00000 

5 

a 

0.00000 

5 

1 

-0 .00000 

5 

2 

0.00000 

5 

3 

- 0.00000 

5 

4 

0.00000 

5 

5 

1.00000 


0.10000000000000000 01 
0 . 30246007718594990-13 
- 0 . 3 13890 327871 29 43 D - 13 
0 . 14304214376636390-12 
- 0.902 71224838612 72 0-13 
0 . 66156676096924960-13 
- 0 . 51897879932932320-13 
0.432154312 335342 20-13 
- 0.37404422 9932 80260-13 
0 . 3372302437298913 D — 13 
- 0 . 31389032787129430- 13 
0 . 30246007718594 89 D -13 
0.10000000000000000 01 
0 . 30246007718594890-13 
- 0.3030101422 29967 70-12 
0 . 14304214378636390-12 
- 0 . 90271224838612720-13 
0 . 66156676096924960-13 
- 0 . 51897879932932320-13 
0 . 43215431233534220-13 
- 0 . 3740 44 2299 328028 D - 13 
0 . 33723024372989130-13 
- 0 . 31389032787129430-13 
0 . 30246007718594890-13 
0.10000000000000000 01 


SSTOP 
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1 

2 

3 

4 

5 


7 

9 

1 0 
11 
i 2 
13 


SJG8 'RUSSELL* .TIM:i=30.PAG2~=75 
C 

C PROGRAM C 3 

C 

C ESTIMATED SPECTRAL AMPLITUDES USING KrONcCKSR CELT* 
C 

C THIS routine evaluates the approximation to the 
C KRCNECKER DELTA FUNCTION AND USci IT TJ CALCULATE 

C THE ESTIMATcD AMPLITUDE i *HEN THE «^AL AMPLITUDES 

C APE S P ... C I F I cO • DCUHlc PnLOISIUN IS USc. D TO RESOLVE 

C THL EXACT VALUES OF THE CFF -DIAGONAL t-L L Mf NT S ♦ 

C 

Pt_AL T h_T a * AM AT •<>_ lT C P » oN N . DO » DR *T F . tn • OK • P I . DC O 

REAL *o CTRUE( 400 ) .CoUM ,CMPLX( 4C.0 ) 

INTEGER Q,R 

c 

DO 5 1=1 .4C0 

5 CTPUt ( I ) =0 *0 

<, 

C INPUT DATA 

C 

N =1 0 
Ms 3 

TN =3.1 
TF= 1 *0 

CTR Uc ( S )-^ »0C 
CTRUd(7) =0.380 
CTRUE (3 ) = CTRUE( 5 ) 

CTRUE ( 1 )=GTRUe(7) 


1 4 

1 S 
1 6 
1 7 

1 6 

1 9 

2 C 
2 1 
22 


24 

25 


27 
2E 
2 9 

JC 

31 50 


NN=2 *N+1 

D N N =N N 

M^=2«M+l 

P 1=3* 1 415226535897 

PRINT e5 

FCRMAT('l r * 5 X ♦ ' Q ' « 7 X « 'C(Q)* *16X» * C ( Q ) ' */) 

DO 90 7=1. NN 

c=l-N-l 

DQ=a 


CSUM=C .0 

DC 8C L=1.MM 
P=L-M-1 
D n =F. 

AMAT = C «G 
CO 5C K=1,N 
Dk=K 

THe.T A =2, «PI*DK*TN* ( O 0/ T N-DR/TF ) /ON.x 
AMAT=OCOS(TH£TA) +A.MAT 
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c 


32 

r 

0 £ LT9t; = ( l. +2.*Af/AT)/CM^ 

33 

L 

so 

CSUMsCTPUt (L ) fCCLTPO +CSUM 

34 


CVPL X( 1 ) =CS6V 

35 

90 

JC INT 5 1 *Q , CMPLX ( I ) , CMFL X( I) 

36 

91 

FOPMAT ( I 8 • F 10.3*028.16.) 

37 


STOP 

36 




sentry 


0 

C(O) 

ceel 

-10 

O.oe4 

0. 84 34 0 7 36 4 3 4789 6 7 D-01 

-9 

0 . 3o6 

0. 255739 1 1 77692324C 00 

-8 

-o.oae; 

-0.69 1659 035 1956? 340-01 

-7 

0.042 

0. 4 2 0989 53C4424638C— 0 1 

-6 

-0.010 

-0.10408102867404310- Cl 

-5 

-0.032 

-0.3173551090331816C-01 

-4 

0.149 

0. 146576640936731 CD 00 

-3 

2.036 

0.20378? 14713460040 Cl 

-2 

-0.253 

-0. 25347770405849830 00 

- I 

0 .17<s 

0. 1764623 163992C86C CC 

0 

-0.161 

-0. 1613439757366 1890 00 

1 

0.176 

0. 1764823 1 639920860 CO 

2 

-0 .253 

-0. 25347770405645830 DC 

.3 

2. 0 3 F 

0. 203762 1471 3460040 01 

4 

0.149 

0. 148S7664C52c73100 CC 

5 

-0.032 

-0 .317355 10902316 160-01 

6 

-0.010 

-0. 10408 102367404310-01 

7 

0.042 

0. 420969 63C4424t38D-01 

8 

-0 .089 

-0.69 165903519962340-01 

Q 

0 . 356 

0.25573511776923240 00 

10 

0.034 

0.04340736434769670-01 


S5T OP 
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iJCfa • RUSSELL T 2 Me =3C , PaGE$ = 7S 
C 

C FRO SPAM 0 4 

C 

C DISCRETE FOURIER TRANSFORM ANALYSIS 


C 

C 

C 

C 

N_ 

r 

C 

1 

2 

3 

4 

5 2 
C 
C 
C 

6 
7 
e 
r 

10 
1 1 
12 

C 
1 3 

14 

15 

C 

C 

C 

1 6 

17 3 

IE 

lC; 6 

20 

2 1 

2 2 10 

C 
C 

c 

c 

c 

c 

c 


THIS PROGRAM CAN EE USCC TC GENERATE A DIGITAL 
FOURIER TRANSFORM FROM TIME SERIES DATA. IT WILL 
ALSO DC A FIPST-CFCSR LINEAR REGRESSION ANALYSIS 
ON THE INPLT DATA TC REVIVE THE CC-VALU6 ANC ANY 
LINEAR TREND. THE hSTIMATEO FOURIER COEFFICIENTS 
A Re. THEN USEC TC COMPUTE AN ESTIMATED TIME SERIcS. 

C I ■IF NS I UN FVALUE (2C1 )» F S A VE ( 20 1 ) *F X ( E04} ,A( 4 00 ) 

D I MC NS I CN CMAGM 400 ) .CPF ASN<40C ) ,F (4 00 )• YF{ 20 1 ) 

00 2 1=1.400 
A ( I ) =0 .0 
8< I ) =0.0 

INPUT' CATA VALUES 

L I NR EG = 1 
NS = 4 1 
TDjiL =6 0 * 

TF =1.0 
FnS=NS 
T N =F NS*TDE L 
SCALE = 52.6 


NLS=NS 

NM AX= ( NS~ l )/2 

CCV = 0 , 0 ' 


READ DATA CARDS AND PRINT CATA VALUES 

RE *D (5 .3) ( FVALUE ( 1 ) . I = 1 . NS ) 

FORM AT < 10X,F 14.2 ) 

PRINT 6 

FORMAT (• I ' .llX.'K* -7X • • T I ME • • 1 3X. - F (T I ME)*/ ) 

DC 1 0 1= 1 . NS 

xdelt a=i*tcel 

PP I NT. I ‘XDeLTa *F VALUF ( I ) 

DC 4 FIPST-CPQER LINEAR REGRESSION ANALYSIS TO 
REMOVE THE CC-VALLE AND ANY LINEAR TREND. 

IF (_INRFG = 0, THE LINEAR REGRESSION IS SKIPPED. 

3 ZERO IS THE Y — INTERCEPT , El IS THE SLOPE. ANC YBAR 
IS THE AVERAGE OF THE DATA VALUES. 
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23 

24 

25 

26 
27 
2 S 

25 

30 

J 1 

33 

34 

35 

36 

37 
36 
35 

40 

41 

42 

43 

44 

45 


46 

47 
4e 
45 

50 

51 

52 


53 

54 

55 


17 


19 


25 


27 

C 

C 

c 

C 

r 

26 


30 

C 

r 

c 


34 


S 6 
57 


56 

59 

60 
61 


IF (LlNRES) 25,26.17 

ksum=o 

KSUM2=0 

YK SUM = 0. 0 

Y?'JW = 0 .0 

DC 15 K = 1 . NS 

KSLM =K,SLM+ K 

KS JW2-<SU^2+ ) 

YKSUV = YK SU VtK*l"V/iLl£(K ) 

YS UK = YSIM+ F VALLE ( K ) 

XFA?= (TCEL^KSUW J/FNS 
Y8AR = YSL'M/F\S 
XSUM2=(TCtL**2)*K.$LM2 
XYSUM=J0EL* YKSLK 

3 1 = < X YSLM— xeAR *Y8AR tFNS } /( XSON2-PNS* < X£AR**2 ) 
EZSR0=Y £ AR-e 1 *XEAR 
PF I N T 2 5 

FORMAT ( • 1 • , 10X , • EZEPtC • , 10X , • b 1 • , 14X , * YBAC * / ) 

PRINT , S ZER C , 81 ,Y EAR 

00 27 K=1.NS 
YF(<)=fc3EFC + 61*K*TC6L 
F 6 A VE ( < ) =F VALL£( K ) 

FVAL U£< K )=F VALUE ( K )- YF ( K ) 

THE TIME St PISS HAS NC* b££N CCRRECTEC ANY 

L inf.ar trend aso the cc-valle has been removec. 

THf MCCIFIEC TIME SERIES IS NO* PRIMED. 

CGNT IN'Jt 
PRINT 25 

FORMAT ( i I ■ • 1 1 X , • K . . 7X • •TIIY5* . 1 3X • • T I ME ) • / } 

CO 30 1=1. NS 

1 I =1 — NM AX- 1 

XL EL T A = I I * TOtL 

PR I NT , I 1 , XCELTA, F VALUE! I ) 

COMPUTE COMPLEX FOURIER AMPLITUDES, ECUATICN(12D.5> 

CALL LNSFTK ( F VALLE «A ,E .NLS ,CCV ) 

PRINT 34 

FORMAT ( *L • ,4X . IN • ,2X • • DC — VALUE I . 5X . * CM AG 1 . IOX 
I • *CPH ASEi , 6X • •FREG-HZ* ,3X . • CM A O C E i / » 

NLMAX = ( NLS— l )/ 2 
DC 35 ! =1 t NLMAX 
XI =1 

CMAG=( <A( I )*A( I) + 3 ( I )*e<I))**C.S)/2. 

CM-CMAC/SCALE 
CMAG0E=20. *ALCG1 O(CM) 
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6 2 


A T E S T “8 ( 1 ) / A ( I ) 

63 


ATE ST = ATEST*ATcS T 

6 4 


ATEST=ATEST*YO. 5 

65 


AT^I OO'J .-A TEST 

6 6 


I F ( A 7 ) 20,20.21 

67 

20 

CPHASE= . S88SS888 

63 


GO T.J 22 

69 

21 

CCNT INuc 

70 


bA=- 3(1 )/ A ( 1 ) 

71 


CFWA3£ = AT AN ( £ A ) 

72 


CPHA 3E = C?HA?S*57 « 29577 

73 


FNLS-NLS 

74 

22 

FROQ = X 1/ (TDElMFNlS ) 

7 5 


CMAGN ( 1 ) =CMAP 

7 o 


CFHASN ( I )= CPHASK 

77 

35 

PR I NT So, I .CCV.CMAC.CFHASS.FPec.CVAGDR 

7 3 

36 

FORM AT( 16. 46 12.3. F 10.2) 


c 

C THf TGTAL PC»£P IN THE SIGNAL IS NCw C£ TE EM INED er 

C SUMMING THE SQLAPF.S OF T ME COMPLEX AMPLITUDES. ThE 

C COMPLEX AMPLITUDES ARE SCALED BY THE SQUARE ROOT 

C OF THIS TOTAL FCWF.F SC THAT AlL AMPLITUDES CAN EE 

C REFERENCED TO THF. TOTAL POKER* THIS NORMALIZATION 

C WILL HELP IN SELECTING THE MCST STATISTICALLY 

C SIGNIFICANT SPECTPAL COMPONENTS. 

C 

79 PRINT 43 

80 43 FORM ATC • I’ ,4X, * N I . i X » 'MAGnIT LOE * I SX . * PHASE * » 3X 

i ,♦ frequency* , ax, • ce relative to*/ 

2 . 4 8 X » * L'N I T Y reT A L F C * E F ' / ) 

81 TP wR = 0 . 0 

82 DC 40 J=1 . NL MAX 

63 40 TP*F = TPWR+CMAGN ( J) ♦CMAGM J1 

44 TPWR = TPWR**0 . 5 

85 Du 4b J =1 . NL M AX 

66 C VAGN ( J) =C MAGN ( J ) / T P»F 

87 X J =J 

88 FC(-q = X J/ (TUf L*FNS ) 

89 CD3 = 20.*-ALCG 1 0(C WAGN ( J ) > 

90 43 PRINT 46 . J , CMAGN ( J ) . CPHASN ( J ) , FPLC .CDE 

91 46 F CRMAT < 16 . 3t 1 3.3 , F 1 0.2 ) 

C 

C COMPUTE THE ESTIMATED TIME SERIES. FO L A 7 1 CN ( 1 2D . 6 ) 

C 

NS=4YNS-3 
T DEL =TCEL/4 
NMAX=< NS-1 )/2 
TF =TN 


94 
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<?e> CALL FSHS ( F X , A , L ,CCV ,NS, TOEL , Tf ) 

57 PRINT 90 

?c 90 FORM AT( • i • • 11 X • • <• .7 X . . T I ME • • 1 IX 

1,'SSTIMATCC F (T IM5 ) •/ > / 

95 DC 10 0 1=1 »NS 

100 11 = I — NM A X - 1 

101 XDEL TA=II7TDEL 

10 2 100 OP IN T i I I i XDt L T A * F X ( I ) 

103 501 STOP 

104 END 

C LINE SPECTRUM SLBRCLTINF 

C 

C THIS SLSfiOUTINfc C-cNERaTES Uc FOURIER C CEFF I C 1 1 NT S 
C A ANC a FCP NLS SAMPLES of the function, f value • 

c 

105 SU3RCU TINT LNSRT V ( F V A LU£ . A , S , NLS , C CV ) 

106 DIMENSION F VAL LE ( 20 1 ) , A( 400) I e (400) 

107 NLMAX= (NLS -1 ) / 2 

10° P I =3 . 14 1 5 ; 26 

10-9 X N N =N LS 

110 D C V =0 . 

111 DO I 5 J= 1 , NLS 

112 15 DCV=OCV + FVaLUE < J ) 

1 13 DC V=QC V/XNN 

1 l A DO 1 6 J =1 i NL o 

113 lb FVAL'Jc ( J )=FVALUL{ J >-CCV 

HE DO 10 Nil, M M AX 

117 A SUM = 0 • 

118 esuM=o. 

115 XN =N 

120 DQ 3 J =1 .N L S 

121 J J =J-NLM AX-1 

122 X J J =J J 

123 ARG=2.0*Pl *XN*XJJ/XNN 

124 ASUM=CCS(AfiG)*FVALLt ( J)+ASU M 

1 26 5 BSUM=$IN(ARG)*FVALLE< J)+6SLM 

126 A(N)=2.0*ASUM/XNN 

127 10 8 ( N) =2. 0*d SUN/XNN 

128 CO 1? J=1,NLS 

129 17 FVALUE ( J ) =F VALUE ( J ) + CCV 

130 RETURN 

131 END 
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c FUUFILP UtPICS SUBROUTINE 

c 

C THIS S Uf3P0U T I Nc Gc Ncf-' ATfcS SAMPLES CF THE DATA 
C FUNCTION. F X ( I J » USINC Tt-c FOUFIEP COEFFICIENTS. 

C A(I), B ( I ) , AND DC-VALUE. CCv, ANC THE NUMBER OF 

C DtS IF EC SAMPLES. NS. TCEL IS THE SAMPLING PEPiJD 
C AND TF IS THE FCUFIFF PEPICC. 

C 


1 32 


SUHFOUTINE F5RS C FX »A .B.LCV, NS.TDFL . 

132 


DIMENSION F X ( £ 04 ) , A ( <; 00 ) »E{ 40 0 ) 

13* 


NM 4X= (NS-1 )/2 

135 


PI =3.1415926 

13fc 


XNS=NS 

137 


CO 10 1=1, NS 

1 36 


K=I-NVAX-1 

139 


XK=K 

1 AO 


FXK= 0 . 

1 41 


DC 5 N= 1 , NMAX 

1A2 


XN=N 

14? 


AP G= ( 2 • 0 »P I*XN»XK*TDEL >/?F 

144 

5 

F X K=FXK+A CM *CCS { APG >+E( N > *SIN (APG ) 

145 

10 

FXC I ) = FXK +• DC V 

146 


RETURN 


SENTRY 


Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 
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K 
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F ( T I M E ) 
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-c 
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03 
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02 
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03 

0 

,7 j 63 0 0 0 E 

02 

1 6 
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0 3 

c 

. 1 4 2 7 5 C 08 

03 

17 

0.1 020 0 00 F 

04 

0 

, 1 360 6008 

C 3 

18 

0.1080000E 

04 

0 

.195770C E 

03 

19 
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04 

0 

. 14 7860 C6 

03 

20 
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04 

0 
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02 

21 
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04 

0 

1 4 36 0 0 OF 

02 

22 
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04 
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02 

23 
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04 

0 
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02 

24 
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04 

c 
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02 

25 

0.1500000 6 

04 

0 
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C3 

26 
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04 

(1 
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03 

27 

0 . 1620000E 

04 

0 
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C3 

28 
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0 
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C 3 

29 

0. 1740000E 

04 

c 

. 74 78C0CE 

02 
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c 
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02 
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04 

0 
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03 
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04 

0 
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03 

35 

0.2100000E 

04 

c 
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G 3 

36 
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04 

c 
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03 

37 

0.2220C 00 E 

C 4 

0 

. 1 1 55 0CCF. 

03 

38 

o .228000C F 

04 
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02 

35 
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03 
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04 

0 
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03 

41 
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04 
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03 


/ 
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BZERG t 1 YEAR 

-0. £33431 7E 02 C « 1 25 9 5 5 35 CO C.6?9o04ot 02 


K r I ft. e F < T I M fc ) 


20 

- 0 . 1 2C0000E 
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-0 .10 200008 

04 

- C. 280465 68 

C 2 

■16 

-0.9600000F 

OJ 
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00 
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02 

1 
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02 

-C.126357EE 

C 3 

2 
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03 
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02 

3 
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03 
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02 
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0 3 

C . 98 16032E 

02 
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6 
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03 
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03 
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OP 
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17 
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f 

DC-VALUE 

CIVAG 


(.PHASE 

FFE C-t-Z 

CMAG— D6 

i 

0.662E -04 

0 .269 E: 

02 

C •344fe. 

0? 

C.4C7F-03 

-6* £3 

? 

0. 662 E - 04 

0. 2406 

02 

-0. 9056 

01 

0.813 E— 03 

-6 .49 


0 .662E-04 

0 .245? 

02 

-0.72 IE. 

C 2 

C .122E-02 

-6. 64 

4 

0.66ZE -04 
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02 
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02 
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C2 
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MAGNITUDE 
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K 

T 1 M6 
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30 

— 0 .45COOOOF 

03 

-0. 665541 1£ 

C 2 

25 

-0.4350000E 

03 

-0.798976 IF 

02 

28 

-C.4200CCOE 

03 

-C.63C451S6 

02 

27 

-0 *40500005 

03 

-0 .39472735 

C 2 

26 

-C.3900000F 

03 

-0.1150C81E . 

02 

25 

-0.37SCCCCF 

03 

0. 17 6 4 6 7 56 

C 2 

24 

-0.3600000F 

03 

0.456 1 2446 

C 2 

23 

-0 *34500005 

03 

0.6926053F 

02 

22 

-0 . 3 3 C 0 OCOF 

C 3 

C. fc?C8726fc 

C 2 

? 1 

-0 *31500005 

03 

0.57644756 

C 2 

20 

-0.3000000E . 

03 

0. 1 0 1 o 1 60 C 

03 

1 5 

-0 .2350000F 

03 

C. 554600 25 

C 2 

18 

-0.2700000E 

03 

0.539849 16 

C 2 

17 

-0.2550000E . 

03 

C. £8772066 

02 

16 

-0 *240 00 00 E 

03 

C. 6772876E 

C 

15 

-0*2 2 50 0 0 0 1 

03 

0 • 5354 0 0 7 E 

02 

14 

-0.21C0000E 

03 

C • 1 0625326 

C 3 

1 3 

-0. 1950000[ 

03 

0 . 12261406 

02 

12 

-0.19C 0 0 ooE 

03 

0. 13726076 

03 

1 1 

-0. *16500005 

03 

0. 14 2 12 10E 

02 

10 

-0.1 500000E 

0 3 

0 • 1355520E 

03 

-9 

-0.13500C 0 E 

03 

C. 1 1 36665E 

03 

-8 

-0.1200000E 

0 3 

C .6120360E 

02 

-7 

-0. 1 050000F 

U3 

C.4506515E 

02 

-6 

-0. 500C0006 

02 

0. 12S4637E 

02 

-5 

-0.7500000c 

02 

-0 . 1006 1 1 £6 

02 

- 4 

-0.600C 0 0 0 E 

02 

-0. 2320346E 

02 

-3 

-0 .4500000F 

0? 

-C.3018206F 

02 

-2 

-0.3000000E 

0 2 

-0.37057 1 26 

02 

-1 

-0.1500000£ 

02 

-C. 45250516 

02 

0 

0 .OOOOOOOE 

OC 

-0. 685804 4E 

02 

1 

0.15000 CO 5 

02 

-0.5217152F 

02 

2 

0.3COOOOO E 

02 

- O. 11361 1 £E 

03 

3 

0.4500000£ 

02 

-0 . 12663426 

02 

4 

c. oa 000 OOf 

02 

-0. 12635e IE 

03 

5 

0 .7SOUOOOL 

02 

- C. 11231188 

0? 

6 

0.50000005 

02 

-0.9232533E 

C 2 

7 

0.10S000CE 

03 

-C. 705453SE 

02 

p 

0 . 1 2000 00 E 

03 

-C .542045 16 

C 2 

5 

0. 1 3500008 

03 

-0.4586327C 

02 

10 

0.15000COE 

03 

— C. 4337 1586 

C 2 

11 

0.1 650000 E 

03 

-0 .40598696 

02 

12 

O.IBCOOOOf 

03 

-0. 32 1 22 1 5F 

02 

13 

0.1950000E 

03 

-C. 122355 6E 

C 2 
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14 

0.2100000£ 

0 2 

0. 1908424F 

0 2 

15 

0. 22500 COE 

03 

r.58C3?79h 

C 2 

1 6 

0 .2*000006 

03 

C *98 160 12E 

02 

17 

0 .25500 OOF 

03 

C . 132843 1 8 

03 

18 

C.27COOOOF 

02 

Oc .V574E74E 

03 

i y 

0 .2850000 2 

03 

c . 17C5720E 

02 

20 

C.3C00000E 

03 

0. 1 732826E 

Oo 

21 

0 .31500003 

02 

C. 168 1662E 

02 

22 

0 .3300000 F 

0 3 

0 . 1576 1 1 OE 

0 3 

23 

0.3450000E . 

0 3 

0. 14291295 

Ob 

24 

0.3600000E 

0 2 

C. 12425555 

C 2 

25 

0.37500 ceE 

03 

0. 1014147E 

03 

26 

o. 3C ; 000 OOE 

0 3 

0.74 6 54 8 6 E 

02 

27 

0.40S00 00 [ 

0 2 

0 . 45248295. 

C2 

2 5 

0.4200000F 

03 

0. 1 5 34 9 0 6£ 

02 

29 

o .4350000L - 

0 3 

-C. 12 6 81 C 9 5 

C2 

30 

0.45COOOOE 

03 

-0. 3707782F 

02 

31 

0. 46500006 

03 

-0. 5718835E 

02 

32 

0.4800Q00F 

03 

-0.734376c£ 

02 

33 

0.49500005 

03 

-0 .66820 t 8E 

02 

34 

O.5100CIOOF 

C 3 

- C. 982097 9E 

C 2 

35 

0 .52500005 

0 3 

-0 . 107t407E 

03 

3 6 

C. 54COGOOE 

03 

-0.1151701E . 

03 

37 

0.55500 COE 

0 3 

-0. 1 191 625E 

03 

38 

C .57000008 

03 

-0 . 1 1868 125 

02 

39 

0. 6350000E 

03 

-0.1129714E 

03 

40 

0 .6GQ0QQ0F 

03 

-0. 1 0 1 8 63 8E 

C 3 

4 1 

0 .61500002 

03 

-0.8S82065E 

02 

42 

C. 6300000F 

03 

-C. 65854C3E 

02 

43 

0 .6450000E 

03 

-C .4 34 16 2 i t 

C 2 

44 

0. cbOOOOOE 

03 

-0.202914 IE 

02 

45 

O.67500COE 

02 

0. 1 567655E 

Cl 

46 

0 .6900000 5 

u 3 

0 .2030936E 

02 

47 

0.7050000E 

03 

0. 34553636 

02 

4 8 

0 .720000GF 

03 

C. 4375 197E 

C 2 

49 

O . 7350000f 

03 

0 .4835826E 

02 

50 

0.7'5000C 0 E 

03 

C.4968050E 

C 2 

51 

0 ,7b50000E 

03 

C . 4940 1 60E 

02 

52 

O.7S00000E 

03 

0 . 4S94465E 

02 

53 

0. 7 9500 OOE 

03 

0. 4893327E 

C2 

54 

u . o 100000E 

0 3 

0 .48985 205 

02 

55 

0.8250000E 

03 

0.4791693E 

02 

56 

0.S4CCC00E 

03 

C. 442465 95 

C2 

57 

0 .85500 OOE 

03 

0. 267S340E 

02 

58 

0.8700001)£ 

03 

0. 2486692E 

02 

t>9 

0 .88500008 

03 

C.6707854E 

Cl 

60 

0 .90000coe 

03 

-0. 1 094993E 

02 

61 

0.9150000E 

03 

-0. 32980515 

02 
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62 

0.9300000F 

03 

-0 .55974726 

02 

63 

0. 9450000F 

02 

-C. 75266266 

02 

64 

0 .9b00000fc 

03 

- C. c7?7 6966 

C 2 

65 

O. 97500 00E. 

03 

-0 • 1 1320276 

03 

6 C 

0. 990C0COE 

03 

-0. 12 24 7 9 26 

03 

67 

0.100500^6 

04 

-C . 1252 1 C 76 

C 2 

66 

0. 1020000E 

04 

-0. 1225146? 

u3 

69 

0. 1 C 350005 

04 

- C . 1 1 b 0 6 3 06 

03 

70 

0.1050000£ 

04 

-0 . 10632256 

02 

71 

0. 1 06 5 0 0 OE 

04 

-C. 1009923E 

03 

72 

0. 1 08 0 0 00 f 

04 

- C • 9 4 C 4 1 76E 

C2 

73 

0.1 095000 t 

04 

-0.85546496 

02 

74 

0.1 1 lOOOOE 

04 

-C. 723781 46 

02 

75 

0 . 1 1 2 60 00 E 

04 

-0 * 5 1/66 4 CE, 

0 2 

76 

0.1140 COOt 

04 

-J .230400 it 

02 

77 

0.1 155C00F 

04 

0 . 1 1 29 £ C 56 

02 

78 

0 . 1 1 70000 F 

04 

0 .45764766 

02 

79 

0 . 1185000E 

04 

C . 73457b 36 

02 

80 

0 . 1 2000005 

04 

C. 68282846 

C2 


SSTOP 
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1 

2 

3 

4 


5 

e 


7 

8 


9 

10 


11 

12 

13 


$J 0 B eRUSSELL'.TIM E =30, PAGES =7S 
C PROGRAM 05 

C 

C FAST FOURIER TRANSFORM SPECTRAL DENSITY 

C 

C THIS PROGRAM ILLUSTRATES A SIMPLE EXAMPLE FOR THE 

C USE OF THE FAST FOURIER TRANSFORM. SUBROUTINE FFAST 

C CAN BE USED IN OTHER PROGRAMS WHERE HIGH SPEED 

C PROCESSING IS NEEDED. THE SUBROUTINE CONTAINS A 

C ZERO FILLING ROUTINE TO ACCOMODATE ANY LENGTH DATA 

C AND ANY DESIRED DISPLAY RESOLUTION. 

C 

DIMENSION All OP. 4), 8(1024), C MAGI 10 24) 

0 I ME NS I ON F MAGI 102 4 ) ,FPHASE( 1024),FR(1024) 

DIMENSION S(1024),SN A M E ( S ) ,X X i I 024) 

0 I ME NS I ON X ( 102 4 ) , XMDC( 1 024 ) , XS V( 1024 ) , XT I ME ( 1 024 ) 

C 

C READ INPUT DATA -- THE TIME SERIES, x( I ) , THE 

C NUMBER OF DATA SAMPLES, NS, THE SAMPLING INTERVAL. 

C DELTA-T. THE DESIRED F FT RESOLUTION OPTION, NRESL. 

C THE DC-REMOVAL OPTION, NDC, AND THE PRINT OPTIONS. 

C NPT1 , NPT 2 , NPT3. NPT4. 

4 READ(S,S,END=SOI} N PTI,N PT2.N PT3.N PT4.N DC 
+, NRESL, NS. XTDELT. SNAM E 

5 FORMAT ( 411. 16, 215, El 8. 7, UX, 5A4 ) 

C 

C **** USER SPECIFIED READ FORMAT **** 

C 

READ ( 5 , 1 0) ( X( I > , 1 = 1 ,NS ) 

10 FORM AT(E14.7} 

C 

C *fr***********$**:<r***44****$*#**A*4**:|c*s<£ 

c 

C PRINT INPUT DATA 

C 

PRINT IS.NPTI.NPT2.NPT3.NPT4.NDC. 

+ NRESL. NS. XTDELT 

15 FORM AT('I',17X,'FA5T F 0 U R I E R T R A N SF 0 R M '. 

+IX ,'SDECTRAL ANALY SIS7//.10X .'PT0PT',3X . 

+ 'NDC‘ , 3X , ‘NRESL* , 3X , ' NS * , 6X , • TDELT-SEC. ’ //, 1 OX, 

+41 I.SX .11,21 7 . E 18.7///) 

C 

C COMPUTE THE AVERAGE (OR DC-VALUE ) , THE MEAN-SQUARE. 

C THE VARIANCE. AND THE STANDARD DEVIATION. 

C 

XN S=PLOAT (N S ) 

DC V =0 .0 
X B A R 2=0 .0 
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14 00 20 1=1, NS 

15 DCV =DCV +X C I } 

16 20 X3AR2-XBAR2+XCI ) FX ( I) 

17 DCV =DCV /X N S 

18 XBA R2=XBAR2/X NS 

19 XVAR=(XdAR2-OCV*OCV)*XN S/(X N S-l .) 

20 STDEV=XVAR**0 .5 
C 

C COMPUTE THE ZERO-MEAN DATA FUNCTION AND TRANSFORM 

C THE DATA TO STANDARD VARIABLES 8Y DIVIDING EACH 

C ZERO-MEAN DATA VALUE BY THE STANDARD DEVIATION. 

C 

21 DO 25 1=1 ,NS 

22 X TIME (I )=FLOAT( l-l)*XTOELT 

23 XMOC ( I ) =Xl I ) -DC V 

24 25 XSV( I } = XMDC ( I ) / STDLV 

C 

25 PRINT 30,SNAM E,DCV,XBAR2,XVAR,STDEV 

26 30 FORMAT!' ',///, 1 OX INPUT DATA FUNCTION STATISTICS:' 

+,2X ,5A4,//,8X , 'AVERAGE', 5X,'M EA N-SOU A REI,5X , 

+'V A Rl A NCE',5X, 'STANDARD DEVIATION ',//,5X ,E 12.5 
+ , 3X.E12. 5.3X.E12.5.3X.E12.5,///) 

C 


27 


1 F (N PT 1 .EO. 0) GO TO 50 

28 


PRINT 35 

29 

35 

FORM AT(' ',7X ,'r',3X /'INPUT data FUNCTIGN 
+,'ZERO M EANI.5X, 'STANDARD V A R 1 A S L E S ',/) 

30 


DO 40 1=1, NS 

31 

40 

PRINT 45.I,X (1} ,XM D C (1 }.X SV (1 ) 

32 

45 

FORM ATC' ' » I 8 * 3E 18.7) 


C 

C SELECT THE DESIRED FORM OF THE DATA FUNCTION 

C FOR FFT PROCESSING. 


C 


33 

50 

N OC =N D C -1 

34 


IF(NDC) 75,55,65 

35 

55 

DO 60 1=1 .NS 

36 

60 

X U ) =X M 0 C ( 1 ) 

37 


GO TO 75 

38 

65 

DO 70 1=1, NS 

39 

70 

X ( [) =XSV ( I ) 


C 

C THE INPUT TIME SERIES HAS SEEN PREPROCESSED INTO 

C A CONVIENENT FORM. THE FFT COEFFICIENTS ARE NOW 

C COMPUTED. 

C 

C FILL THE FFT ARRAY WITH TIME SERIES DATA AND 

C COMPUTE THE REAL AND IMAGINARY COMPONENTS OF THE 

C COMPLEX FOURIER AMPLITUDE COEFFICIENTS. 

C 
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40 75 CONTINUE 

4 1 DO 80 1=1. NS 

42 A ( I } =X ( I ) 

43 80 B(l}=0.0 

44 K=0 

45 CALL FFAST(A,B,NS,NRESL,K) 

46 N=2** ( K+NRESL > 


47 


48 

49 

50 


51 

52 

53 

54 

55 

56 

57 


58 

59 

60 


61 

62 

63 

64 

65 

66 

67 

68 
69 


K MAX= 2* 4 ( K + NRESL— 1 > 

C 

C CONVERT THE REAL ANO IMAGINARY COMPONENTS INTO 

C .MAGNITUDE AND PHASE COMPONENTS. 

C 

DO 85 1=1, N 

F MAG ( I)=(A(I)*A(I)+8(I>*B(in**0.5 
85 FPHASE(I)=5 7.29577*ATA N 2(B (I ),A (!» 

C 

C COMMUTE THE ESTIMATED DISCRETE FOURIER AMPLITUDE 

C SPECTRUM BY SCALING. 

C 

XN=FLOAT(N) 

DO 90 I =1.K M A X 
L =1-1 

XL=FLOA T ( L ) 

XX( I )=FMAG< I ) 4XTDELT 
C MAG ( I > -=F MA G ( I ) / XN 
90 FR(!)=XL/(XN*XTDELT) 

C 

C COMPUTE THE ESTIMATEO POWER SPECTRAL DENSITY 

C FUNCTION* 

C 

DO 95 1=1 »KMAX 
L =1-1 

95 S(I)=CMAG(I>*CMAG(I)*XTDELT*XN 

C 

C PRINT RESULTS 

C 

I F ( N PT 2 .EO. 0 ) GO TO 115 
PRINT 100 

100 FORM AT(IIU0X , I F FT COMPLEX OUTPUT',//,9x,'N '.6X , 

+'FM AG',7X ,'FPHASE',6X ,'A(N )',8X ,'B(N )'./) 

DO 105 I =1 , N 

105 PRINT 110,1, FM A G (I ), F PH A SE « (), A (I ), B ( I ) 

110 FORMAT ( I 11 I 9 » 4E 12 *3 > 

C 

115 I F ( N PT 3 .EO. 0) GO TO 135 

PRINT 120 

120 FORMAT! • 1 l OX, ’ AMPLITUDE SPECTRAL DENSITY',//, 
+9X .'L',6X ,'FREOU EN CY - HZ',5X 
+ , • MAG XX (W'-V/HZ',7X ,'CM A G -D FT ',/) 
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70 DO IZ5 1 = 1* KM AX 

71 L =1-1 

72 125 PRINT 130.L .F R ( I) ,XX( I) ,C M A G (I ) 

73 130 FORMAT!' ' . I 9.3E 18. 7) 

74 135 IFCNPT4 .EO. 0) GO TO 499 

75 PRINT 140 

76 140 FORM AT(T,10X, 'POWER SPECTRAL DEN SITY ’ ( //.9X ,’L 

+ 7X , • FR£QU£NCY-HZ • , 6X, ' MAG SCI) - W /HZ * ,//) 

77 00145 I =1 . K M AX 

78 L =1 - 1 

79 145 PRINT 150.L ,FR( I) *S( I , 

80 150 FORM AT ( ' , , I 9 • 2E 1 9 • 7) 

81 499 CaNTINUE 

C 

C **** USER SUPPLIED PLOT ROUTINES **** 

C 

c aa INPUTS aa 

c 

C TIM E VARIABLE, XT I ME ( I ) . 

C INPUT DATA FUNCT ION, X {I ). 

C DATA FUNCTION WITH DC-VALUE REMOVED, XMDCCI). 

C STANDARD VARIABLES, XSV(I). 

C 

c aa OUTPUTS as 

c 

c FREQUENCY VARIABLE IN HERTZ, FR{I). 

C AMPLITUDE SPECTRAL DENSITY, XX(I). 

C POWER SPECTRAL DENSITY. SCI). 

C 

82 GO TO 4 

83 501 STOP 

84 END 

C 

C FAST FOURIER TRANSFORM SUBROUTINE 

C 

85 SU8ROUTINE F F A ST (A ,B ,N S,N R E SL ,K ) 

C 

86 DIMENSION A (1024). 8(1024) 

C 

c TEST THE TOTAL NUMBER OF POINTS AND IF NS IS N.E. 

c TO 2* * K . FILL THE REMAINING TIME SERIES ARRAY, ACI) 

c WITH ZEROS. IF ADDITIONAL DISPLAY RESOLUTION IS 

C DESIRED, ZERO-FILLING CAN BE IMPLEMENTED BY 

C SELECTING THE RESOLUTION CONSTANT. NRESL. 

C 

C 

c DETERMINE THE P0WER-0F-TW0 ORDER, K , 

C 
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87 

88 

89 

90 

91 

92 


93 

95 

96 


97 

98 

99 
100 
101 
102 
103 
1 04 

105 

106 

107 

108 
109 
1 lO 
111 
112 
113 
1 14 

115 

116 

117 

118 
1 19 
120 
121 
122 
123 
1 24 

125 

126 

127 

128 
129 


ZNS=FL0ATCNSJ 

XK=ALOG(ZNS)/ALDG(2.0) 

K =1 NT(X K ) 

N=2**K 

IF(NS.GT.N) K =K +1 
N =2**(K +N R E SL ) 

C 

C ZERO-FILL. 

C 

NSPI =N S+l 
DO 5 I =N S P I . N 
A {l)=0.0 
5 8 ( 1 )= 0.0 

C 

c COMPUTE jh£ FFT ON N DATA POINTS USING TIME 

c DECOMPOSITION WITH INPUT BIT REVERSAL. 

C 

MR = 0 
NN=N- 1 

DO 20 M=1»NN 
L =N 

10 L=L/2 

I F (M R+L.GT.N N ) GO TO 10 
MR = MCO( MR.L1+L 
IF(MR.LE.M) GO TO 20 
TR=A < M+ 1 ) 

At«H)=A(MR+l) 

A (M R+l )=TR 
T I =B (M+ 1 ) 

B ( M R +1 ) =T I 
20 ceNT INUE 

L =1 

30 IF(L.GE.N) RETURN 

I STEP =2* L 
EL=FLOAT(L) 

DO 4 M= 1 » L 

Z=3.14-15926*FLOAT(l-M )/EL 
W R =C 0 S { Z) 

W I =SI N (Z) 

DO 4 l=M .N.ISTEP 
J =1 +L 

T R =W R* A (J )-W l*B (J ) 

T I =W R * S {J ) +W I * A (J ) 

A (J )=A(I)-TR 
B (J )=B( I >-T I 
A ( I ) =A ( I ) +T R 
4 a (i ) =8( i >+t i 
L =1 ST E P 
GO TO 30 
END 



458 


FAST FOURIER TRANSFORM SPECTRAL ANALYSIS 


PTOPT 

NDC 

NRESL 

NS 

TDELT -SEC. 

1111 


a 

15 

0.3000000 E 00 


INPUT OATA FUNCTION STATISTICS: E X P ( - 1 1 S I N {T ) 

AVERAGE MEAN-SQUARE VARIANCE STANDARD DEVIATION 

0. 11129E 00 0 . 27 733E- 01 0.16443E-C1 0.12623E 00 


T 

INPUT DATA FUNCTION 

ZERO MEAN 

STANDARD VARIA6L 

1 

O.OOOOOOOE OC 

■0.1112922E 

00 

- 3 *86 79 1 57E 

oa 

2 

0.2189268E 00 

C .1 076345E 

00 

0.8393911E 

00 

3 

0.3098824E 00 

0. 1985902 E 

00 

0.1548711E 

01 

4 

0.3184770E 00 

0.2071B 47E 

00 

0.1615736E 

01 

5 

0 ♦ 28 0724SE 00 

0. 1 694326E 

GO 

0.1321324E 

01 

6 

0.2225712E 00 

0.1112790E 

00 

a *8678 120E 

00 

7 

0.1609759E 00 

0.4968363E- 

01 

0 *.3874592E 

00 

8 

0.1057055E CO 

0. 5586743 E - 

02 

-0.4356838=: 

-01 

9 

0.6127660E -0 1 

O.SOOI564E- 

01 

-0 ‘3900484E 

00 

10 

0.2872230E-OI 

0. d256990E - 

01 

-0 ♦ 64 39236E 

00 

11 

0 . 7026002E-02 

a * 1 0 42 662 E 

oa 

- 0. 8131231E 

00 

12 

-0.5818199=:-02 

0. 1171104E 

00 

-0.9132890E 

00 

13 

- 0. 1209 130E - O 1 

•0.1 233835E 

00 

-0.9622099E 

00 

l 4 

-0. 13921 70E-01 

0.1252139E 

00 

- 0 . 9764843E 

00 

1 S 

-0. 1306980E -O 1 

a* 1243620E 

oa 

-0.9698405E 

00 



F FT COMPLEX 

OUTPUT 






N 

F M AG 

FPHASE 

A ( N) 


B(N) 


1 

0.167E 

01 

O.OOOE 

00 

0 .167E 

01 

O.OOOE 

00 

2 

0.127E 

01 

-0.836E 

02 

0.141 E 

CO 

-0.126E 

01 

3 

0. 4 73 E 

00 

-0 .134E 

03 

-0.326E 

00 

-0.343E 

00 

4 

0.226E 

00 

-0.153E 

03 

-0.201E 

00 

-0.103E 

00 

5 

0. 1 40E 

00 

-0.1 64 E 

03 

-0.135E 

00 

-0.379E- 

Ol 

6 

O.105E 

00 

-0.172E 

03 

-0.104E 

00 

-0 . 1 44E- 

-01 

7 

0.887E- 

01 

-0.177E 

03 

-0.886E- 

01 

-0.491E - 

02 
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8 

0.819E-OI 

-0.179E 

03 

-0.819E 

-Ol 

-0. 130E-02 

<5 

0. 7 99 E— 0 1 

0.180E 

03 

-0.799E 

-Ol 

O.OOOE 00 

10 

0.819E-OI 

0.179E 

03 

-0.81 9c 

-01 

0.130E-02 

11 

0.887E-OI 

0.177E 

03 

-0.886 = 

-01 

0.491E-02 

12 

0.105E 00 

0.172E 

03 

-0. 104E 

00 

0.144E-OI 

13 

0 .140 E 00 

0.164E 

03 

-0^135E 

00 

0.379E-OI 

14 

0.226E 00 

0.153E 

03 

-0.201E 

00 

0. 103 E 00 

15 

0.473E 00 

0.134E 

03 

-0.326E 

00 

0.343E 00 

16 

0.127E 01 

0.836E 

02 

0. 141E 

00 

0.126E 01 

AMPLITUDE SPECTRAL DENSITY 




L 

FREQUENCY 

-HZ 

MAG 

X X (W )-V/H Z 

CM AG-DFT 

0 

0.0000000E 

00 

0.5008157E 

00 

0. 1043366E 00 

1 

0.2083333E 

00 

0.3807855E 

00 

0 .7933033E-01 

2 

0.4165666E 

00 

0. 1419684E 

00 

0.2957676E-OI 

3 

0 . 6249999 E 

00 

0*6781 936c— 

01 

0. 1412904E -O 1 

4 

0 .8333333E 

00 

0.4204089E - 

Ol 

0 .S758519E-02 

5 

0. 1041666E 

01 

0.3137103E- 1 

01 

0.6535631E-02 

6 

0 .1249999E 

01 

0.2661949E-I 

01 

0.5545728E -02 

7 

0 .1458333E 

01 

0.2456124E-I 

01 

0.5116925E -02 



POWER 5PECTRA L 

DENSITY 


L 

FREQUENCY 

-HZ 

MAG 5(I)-W/HZ 

0 

O.OOOOOOOE 

00 

0. 5225342 E-OI 

1 

0.2083333E 

00 

0.3020782E -O 1 

2 

0. 4166666E 

00 

0.4198965E -02 

3 

0. 6249999E 

00 

0.9582227E-03 

4 

0.8333333E 

00 

0.3682158E -03 

5 

0. 1041666E 

01 

0.2050294E -03 

6 

0. 1249999E 

01 

0.1476245E -03 

7 

0.1458333E 

01 

0.1256780E-03 
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1 

2 


3 

4 


5 

6 


7 

8 


9 

10 

11 

12 

13 

14 

15 

16 

17 

18 


SJOB ’RUSSELL • , 7 1 ME=3 0 , PA GES=7 5 
C 

C PROGRAM 06 

C 

C ESTIMATION OF S I N G L E -S I N E W A V E PARAMETERS 
C 

C THIS PROGRAM COMPUTES THE BEST FIT OF A SINGLE- 

C FREQUENCY SINEWAVE TO AN INPUT TIME SERIES WHEN 

C THE SINEWAVE PERIOD IS KNOWN OR ESTIMATED. 

C 

DIMENSION X (1000),FSCI000).X RCIOCC).X M DC (IOOO) 
DIMENSION SNAM E(S).XTIM E(1000) 

C 

C READ INPUT PARAMETERS 
C 

4 READ ( 5 ,5 , END=501 ) N S .T Z E R 0 .T D E L T .S N A M E 

5 FORMAT I I 5 * 5X » 2E1 6*6 * 7X * 5A4) 

C 

c A A A A USER SPECIFIED READ FORMAT a**a 

c 

RE AD ( 5 , 1 0 ) ( X ( I ) * I = I >NS ) 

10 FORMAT( E 14.7) 

C 

c AAA AAAAAAAAAAAAAA a$aaa*aa*aaaaaaa a a a 

c 

C PRINT INoUT PARAMETERS 

c 

PRINT 15 ,NS, T2ER O.TDELT 

15 FORM A T ( 'I > ,9X,'ESTIM ATION OF SI N G L E -SI N E W A V E 
+'PA RAM ETERS7//.HX ,'IN PUT PA RAM ETERS:'//,11X 
+ , ’ NS ’ ,5X * » T ZERO- SEC • * » 7X , • TDELT-SEC. ’ // 

+,1 14 , 2el 7.7) 

C 

C COMPUTE THE AVERAGE, MEAN-SQUARE , VARIANCE. AND 
C STANDARD DEVIATION OF THE INPUT TIME SERIES. 

C 

X N S=N S 
DCV =0 .0 
X BA R2=0.0 
DO 20 I =1 .N S 
D C V =D C V +X ( I ) 

20 XBAR2=XBAR2+X(1)*X(1) 

DC V=DCV/XNS 
X8AR2=X3AR2/XNS 

XV4R=(XBAR2-DCV*DCV }*X N S/(X N S-l .) 

STDE V = XVA.RAA0 . 5 
C 

C COMOUTE THE ZERO-MEAN DATA FUNCTION 
C 
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1 9 

20 25 
C 

C 

C 

c 

21 

22 30 


23 

24 31 

25 

26 35 

27 36 
C 

C 

c 

c 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 40 
C 

38 

39 

C 

C 

C 

C 

C 

40 

41 

C 

c 

c 

c 

42 

43 

C 

C 

C 

C 


DO 25 1 = 1 , NS 
X M DC(I)=X (J )- D C V 

PRINT THE- INPUT TIME SERIES STATISTICS. INPUT 
TIME SERIES, ANO IERO-MEAN DATA FUNCTION. 

PRINT 30 » SNAME .OCV , X8AR2 » XVAR , ST DEV 

FORMAT!* '///, 10X ,» INPUT DATA FUNCTION STATISTICS 
+»2X.5A4//,8X, ' AVERAGE * , 4X , ' MEAN- SQUARE ' ,4X 
*, * VARIANCE' , 3X. ' STANDARD OEVI AT I 0N’//3X , 4E14 .5/// ) 
PR INT 31 

FORM AT(' 'iSX.'T' , 4X , • INPUT time SERIES' 

+• ,4X» ' ZERO-MEAN TIM E SERIES'/) 

DO 35 1=1, NS 

PRINT 36 ♦ I .X( I ) * XMDC ( I ) 

FORM AT(I7,EI8.4,E21.4) 

COMPUTE EQUATIONS (1J.15) AND (13.16) USING THE 
ZERO MEAN OATA FUNCTION. 

N =(N S-l)/2 

01=3. 141 5926 

Z£0= ( 2 . 4PI4T0ELT )/TZERO 

AL PH A =0 . 0 

B ETA =0 .0 

DO 40 1=1, NS 

L =1 - N - 1 

X L =L 

ALPH A= AL PHA+ XM DC ( I >*COS( XL*ZED ) 

BETA =B ETA +XM DC(I)*SIN(XL*ZED) 

ALPHA=AL PHA/XNS 
BETA =- BETA /X NS 

COMPUTE THE MAGITUOE AND PHASE OF THE COMPLEX 
AMPLITUDE COEFFICIENT GIVEN BY EOUATIONS 
(13.17) AND (13.18). 

CM AG= ( ALPHA* ALPHA+SET A48ETA ) **0 , 5 
CPHASE=57.29577*ATAN2( BETA , ALPHA ) 

COMPUTE THE MAGNITUDE AND PHASE OF- A SINGLE- 
FREQUENCY SINEWAVE WHICH IS A BEST FIT. 

DM AG =2.* CM AG 
OPHASE=C PHASE-270. 

PRINT THE FREQUENCY, MAGNITUDE, AND PHASE OF THE 
S INEWAVE. 
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4-4 

45 

4-6 45 


C 

C 

C 

c 

*7 

48 

49 

50 

51 50 

52 

53 

54 

55 

5 6 60 

57 

58 

59 

60 

C 

C 

C 

61 

62 70 


C 

c 

c 

c 

63 

64 75 


65 

66 

67 

68 
6S 

70 80 

71 90 
C 

C 

c 

c 

c 


FREQi 1 ./TZERO 
PRINT 45. FREQ. DM AG.DPHASE 
FORM AT('I'.7X .'BEST-FIT s I newave : • //, 7x 
■f.'FREQU ENCY '.7X ,'OM AG',8X . -D P H A S E -0 E G 7/.4X . 

+3£ 14« 4/// ) 

COMPUTE THE -RESIDUE" TIME SERIES AND 
ITS STATISTICS. 

DPHASE=2.*PI*DPHASE/360. 

DO 50 1=1, NS 
L =I-N -I 
XL=L 

XR C I ) =XMOC( I ) -OMAG4SI N( XL*ZED + OPHASE ) 

DCV =0.0 
X BA R2=0.O 
DO 60 1=1, NS 
DCV=DCV+XR( I ) 

X8AR2=X8AR2-(-XR< I ) *XR( I ) 

OCV=OCV/XNS 
X8AR2 = X8AR2/XNS 

XVAR=(XBAR2-0CV*0CV)*XNS/(XNS-I.J 
ST DEV =X VAR** 0.5 

print THE "RES I DU E •* TIME SERIES STATISTICS. 

PRINT 70.OCV.X8AR2.XVAR,STDEV 

FORMAT!* "tOX ,'RESI DU E TIME SERIES STATISTICS:'// 
+ » 8X* ' AVERAGE* «4X* 'MEAN-SQUARE* ,4X. 'VARIANCE* 

+,3X . 'ST A N 0 A RO O E V I A T I O N '//3X .4EI4.5///) 

PRINT THE ZERO-MEAN DATA FUNCTION. BEST FIT 
SINEWAVE. AHO RESIDUE TIME SERIES. 

PRINT 75 

FORMATC 7//.14X, 'REFERENCE'. 26X .-RESIDUE-/. 

+6X ,'T',6X ,'TI M E SERIES* » 7 X i 'SINEfcAVE* 

+ « 8 X « ' TIME SERIES'//) 

DO 80 1=1. NS 
L =1 - N - 1 
X L =L 

XTIM E(l)=FLOAT(l-l)*TOELT 
FS(l)=CMAG*SIN(XL*ZED+0 PHASE) 

PRINT 90.L.X M O C ( I J . F S ( I J ,X R(l) 

FORM AT(I7.3E17.4) 

^ -jlf ■+• ■+• 

**** USER SUPPLIED PLOT ROUTINES 
S3 INPUTS 38 
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7 2 

73 

74 


C TIME VARIABLE, XT I ME ( I ) . 

C INPUT DATA FUNCTION. XCI). 

C ZERO-MEAN DATA FUNCTION. XMOC(I). 

C 

c as outputs as 

c 

c SINEWAVE FUNCTION. FS(I ). 

C ^RESIDUE" TIME SERIFS. XR(I). 

C 

C 

GO TO 4 
50 1 STO© 

END 


ESTIMATION OF SI N G L E -SI N E W A V E PARAMETERS 


INPUT PARAMETERS: 
NS TZERO-SEC. 

41 0.4926101E 


TDELT-SEC. 

03 0.6000000E 02 


0 . 


INPUT DATA FUNCTION STATISTICS: HER A 7/29/75 LINREG 

AVERAGE MEAN-SQUARE VARIANCE STANDARD DEVIATION 
72944E-04 0.14904E 05 0.I5276E 05 0.12360E 03 


T INPUT TIME SERIES 

1 0.1626E 02 

2 - 0.9235E 02 

3 -0.2222E 03 

4 -0.2805E 03 

5 -0.2426E 03 

6 -0.6166E 02 

7 0.1153E 03 

8 0.2187E 03 

0.2S53E 03 


ZERO-MEAN TIME SERIES 

0.1626E 02 
-0.9235E 02 
-0.2222E 03 
-0.280SE 03 
-0.2426: 03 
-0.6166E 02 
0.1153E 03 
0.2187E 03 
0.26S8E 03 


9 
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10 

0.2273E 

03 

0.2273E 

03 

1 1 

O.1097E 

03 

O.1097E 

03 

12 

-0.3037E 

02 

-0 .3037E 

02 

13 

-0.8642E 

02 

-0.8642E 

02 

1 4 

-0 .6305E 

02 

-O.6305E 

02 

15 

0 « 4 56 1 E 

02 

0.4561E 

02 

16 

O.1016E 

03 

O.1016E 

03 

17 

0.8773E 

02 

0.8773E 

02 

18 

0.1373E 

03 

0.1373E 

03 

19 

O.8120E 

02 

O.8120E 

02 

20 

-O.2320E 

02 

-0 .2320E 

02 

21 

- 0. 6858E 

02 

-0.6858E 

02 

22 

-0. 1 264 E 

03 

-0.1 264E 

03 

23 

-0 .5421E 

02 

-0 .542 IE 

02 

24 

-0.3212E 

02 

-0.321 2E 

02 

25 

0.9816E 

02 

0.9816E 

02 

26 

0.1733E 

03 

0, 1 7 33E 

03 

27 

0.1243E 

03 

0.1243E 

03 

28 

0.1535E 

02 

0 .1535E 

02 

29 

- 0 . 7344E 

02 

-0.7344E 

02 

30 

-0 .1152E 

03 

-0.1152E 

03 

31 

- 0.1 019E 

03 

-O.1019E 

03 

32 

-O.2029E 

02 

-0 .2029E 

02 

33 

0.4375E 

02 

0.4375E 

02 

34 

0.4894E 

02 

0 • 4894 E 

02 

35 

0 .4425E 

02 

0.4425E 

02 

36 

-0 .1095E 

02 

-O.1095E 

02 

37 

-0.9798E 

02 

-0.9798E 

02 

38 

-0 .1225E 

03 

-0.1225E 

03 

39 

- 0.9404E 

02 

-O.9404E 

02 

40 

-0 .2304E 

02 

-0 .2304E 

02 

4 1 

0.8838E 

02 

0.8838E 

02 


BEST-FIT SINEWAVE: 
FREQUENCY DMAG 

0.2030E-02 0.1285E 03 


DPHASE-DEG 
-O.1053E 03 


RESIDUE TIME SERIES STATISTICS: 


average MEAN-SQUARE 


VARIANCE STANDARD DEVIATION 


0.15755E 00 


0.66434E 04 


0.6S095E 04 


O.82520E 02 
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REFERENCE RESIDUE 


T 

TIME SERIES 

SINEWAVE 

TIME SERIES 

-20 

0. 1626E 

02 

0.1273E 

03 

-O .1 1 1 O E 

03 

-19 

-0.9235E 

02 

(J '.7988E 

02 

-0 .1722E 

03 

-18 

-0 . 2222E 

03 

-0.1209E 

02 

- 0 . 2 10 1 E 

03 

-17 

-O.2805E 

03 

-0.9731c 

02 

-0. 1832E 

03 

-16 

-0.2426: 

03 

-0.1283E 

03 

-0.1 144E 

03 

-15 

— 0 *61 66E 

02 

-0 .8770E 

02 

O.2604E 

02 

-14 

0.1153E 

03 

0.1774E 

01 

0.113SE 

03 

-13 

0.2187E 

03 

0 .9026E 

02 

0. 1284E 

03 

-12 

0. 2653E 

03 

0.1284E 

03 

0.1374E 

03 

-11 

0.2273E 

03 

0 .9496E 

02 

0.1324E 

03 

-10 

0. 1 0 97E 

03 

0.85S1E 

01 

0.1012E 

03 

-9 

- 0 .3 037E 

02 

-0.8262E 

02 

0.5225E 

02 

-8 

—0 • 8642E 

02 

-0.1277E 

03 

0.4130E 

02 

-7 

-0, 63 05S 

02 

-0 .1016E 

03 

0.385SE 

02 

-6 

0 .4561E 

02 

-0.1882E 

02 

0.6443E 

02 

-5 

0.1016E 

03 

0 .74-46E 

02 

0.2716E 

02 

-4 

0.8773E 

02 

0.1262E 

03 

-0.3848E 

02 

-3 

0 . 1 3 73 E 

03 

o . 1076E 

03 

0.2969E 

02 

-2 

0.81.20E 

02 

0.2897E 

02 

0.5224E 

02 

-1 

-0 . 2320E 

02 

-0.6S81E 

02 

0.4260E 

02 

0 

-0.68S8E 

02 

-0 . 1 2S9E 

03 

0.SS30E 

02 

1 

-0. 12 64E 

03 

-0.1129E 

03 

-0.1348E 

02 

2 

-O.S421E 

02 

-0.3893E 

02 

- 0.IS28E 

02 

3 

-0.3212E 

02 

0 . S873E 

02 

-0.88SSE 

02 

4 

0.9816E 

02 

0.1208E 

03 

-0.2259E 

02 

S 

0.1733E 

03 

0.1 i. 74E 

03 

0.55S4E 

02 

6 

0 . 1243E 

03 

0.4864E 

02 

0.7562E 

02 

7 

0.1535E 

02 

-0.4729E 

02 

0.6264E 

02 

8 

-0 . 7344E 

02 

-0.116SE 

03 

0.4340E 

02 

9 

- 0. 1 152 E 

03 

-0.1212E 

03 

0.6065E 

01 

10 

-0. 1019E 

03 

-0.5803E 

02 

-0.4383E 

02 

11 

-0.2029E 

02 

0.3754E 

02 

-0.5783E 

02 

12 

0.4375E 

02 

0.1122E 

03 

-0.6843E 

02 

13 

0.4894E 

02 

0 . 1243E 

03 

-0 .7531E 

02 

14 

0. 4425E 

02 

0.6705E 

02 

-0 .2280E 

02 

15 

-0. 1095E 

02 

-0.2755E 

02 

0.1660E 

02 

16 

-0.9798E 

02 

-0.1068E 

03 

O.8809E 

01 

17 

-0.1225E 

03 

-C .1265E 

03 

0.3960E 

01 

18 

-0.94 04E 

02 

-0. 7563E 

02 

-0.1841E 

02 

19 

-0.2304E 

02 

0.1738E 

02 

- 0 .4042 E 

02 

20 

0 . 8838E 

02 

0.1007E 

03 

- 0. 1232 E 

02 



466 


5.108 ■RUSSELL* ,TI *E=3 0 , PAGES=75 

C 

C PROGRAM 07 

C 

C MAXIMUM ENTROPY SPECTRAL ANALYSIS 

C 

C THIS PROGRAM COMPUTES THE MAXIMUM ENTROPY SPECTRAL 

C ESTIMATE FOR A TIME SERIES. THE PROGRAM WILL 00 A 

C FIRST ORDER LINEAR REGRESSION ANALYSIS. IF DESIRED, 

C AND ALSO CONVERT THE DATA TO STANDARD VARIABLES. 

C 

1 DIMENSION A K (lOO).FR(lOOO) 

2 DIMENSION S(1000),SPL OT(1000),SN A M E (S) 

3 DIMENSION X (1000), X M DCCIOOO).X NORM (1000) 

4 DIMENSION X SV (1000), XTIN E(1000J 
C 

C READ INPUT DATA -- THE TIME SERIES, XCI), THE 

C NUMBER OF DATA SAMPLES. NS, THE MAXIMUM NUM8ER OF 

C COEFFICIENTS TO BE CALCULATED, MM, THE SAMPLING 

C INTERVAL. OELTA-T. ANO THE DESIRED RESOLUTION IN 

C THE MEM SPECTRAL ESTIMATE, DELTA-F. SNAME IS A USER 

C DEFINED' LABEL FOR THE INPUT DATA SET. LINREG IS 

C A PROCESSING OPTION. NPTI. NPT 2 , NPT3, NPT 4 » ARE 

C PRINT OPTIONS. 

C 

5 4 READ(5,5.ENO=501) N PTI,N PT2.N PT3.N PT4 

+. LINREG. NS,NN.XTDELT,FDELT.S NAME 

6 5 F0RMAT(4I1 , 1 6 ,2 I 5 , E i 3 . 3 , E I S . 3 , 5 A4 ) 

7 N =N S 

8 X N =FLOAT (N ) 

9 IF(FDELT) 501,6,7 

10 6 FDELT=.25/(FLOAT(NS)*XTDELT) 

11 7 CONTINUE 

C 

c **** USER SPECIFIED READ FORMAT **** 

C 

C READ INPUT DATA FUNCTION. 

C 

12 READ(5 , 10K X ( I) ,1=1 ,N ) 

13 10 FORMAT! 1 OX, F 14.2 ) 

C 

C PRINT INPUT DATA. 

C 

14 PRINT 15 

15 15 FORMAT (* 1 *» 1 7X »• MAX I MUM ENTROPY SPECTRAL ANALYSIS', 

+///.9X . -PTO PT - ,2X ,-LIN REG - ,3X ,-N S-,5X . -M M 
+3X, OELTA-T. SEC • ,3X ♦ • DELTA-F , HZ'/) 

16 PRINT 20.NPT1 ,NPT2,NPT3,NPT4, LINREG, NS, MM, XTDELT.FOELT 

17 20 FORMAT!* » ,9 X , 41 1 , 4X , I 1 ♦ 3 X , I 5 , I 7 , E I S . 7 , E 1 6 . 7 ) 
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18 

19 

20 
21 
22 


23 

24 

25 

26 
27 


28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 


41 

42 

43 

44 

45 


c 

C DO A FIRST ORDER LINEAR REGRESSION ANALYSIS TO 

C REMOVE ANY LINEAR TREND AND COMPUTE THE 

C AVERAGE VALUE. 

C 

25 KSU M =0 

K SUM 2 = 0 
X KSU M=0 . 0 
X SU M= 0 • 0 
X D C =0 . 

C 

C COMPUTE THE AVERAGE VALUE. 

C 

DO 30 K=I.N 
30 X D C =X D C +X ( K ) 

X DC=X DC/X N 
DC 35 K =1 ,N 

35 XM DC(K)=X(K)-XDC 

C 

C COMPUTE LINEAR REGRESSION. 

c- 

DO 40 K =1 ,N 
Ksum=ksum+k 
KSUM 2=KSUM 2 +( K * * 2 ) 

XKSUM =XKSUM +FLOAT{K)*X(K) 

40 XSUM=XSUM+X(K ) 

XT BAR = XTDELT 4FL0 AT( KSUM)/XN 
XBAR=XSUM/XN 

XTSUM 2=XTOELT**2*FLOAT(KSUM 2) 

X T SUM = XT DEL T * XX SUM 

BX1=(XTSUN-XTBAR*X6AR*XN)/(XTSUM2-XN*(XTBAR**2.» 

BZER0=XBAR-BX1*XTBAR 

00 45 K =1 ,N 

45 XN0RM(K)=X(K)-BZER0-BX1*K*XT0ELT 

C 

C THE TIME SERIES HAS NOW BEEN CORRECTED FOR ANY 

C LINEAR TREND AND THE DC-VALUE HAS BEEN REMOVCO. 

C 

C 

C COMPUTE THE STATISTICS FOR THE VARIOUS FORMS OF 

C THE DATA FUNCTION. ALSO, TRANSFORM THE DATA INTO 

C STANOARO VARIABLES 8V DIVIDING EACH ZERO-MEAN DATA 

C VALUE BY THE STANOARO DEVIATION. 

C 

50 PZ E R 01=0 .0 

PZ E R 02=0 .0 
PZ E R 03=0 .0 
00 55 1=1, N 

PZER01=PZER01+X(I)*X(I) 
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46 


PZER02=PZER02+XNORM ( I)*XNORM( I) 

47 

55 

PZEQ03=PZER03+(X(!)-XDC)*(X (l)-XDC) 

48 


PT =PZE R01/X N 

49 


PZEROI=PZEROI/(X N-l.) 

50 


PZ ERO 2 -P ZEROS/ ( X N - 1 . ) 

51 


PZER03=PZER03/(X N-l.) 

52 


S'TD£V2 = PZER02**0 .5 

53 


STDEV3=PZER03**O.5 

54 


00 60 ! =1 ,N 

55 


X 1 =! 

56 


X T ! M E (1 )=X l*XTDELT 

57 

60 

r 

X SV ( l)=XNORM (1)/STDEV2 

58 

L 

PRINT 65 » S N A M E » XDC , P T , PZER03 , ST DE V3 

59 

65 

FORMATC • ,///.! OX, ' INPUT DATA FUNCTION STATISTICS: 
+• 2 X * 5 A 4i//»8X» 'AVERAGE ' * 5X , ' MEAN- SQUARE • , 5X , 

+ • VARIANCE* ,4X. 'STANDARD DEVIATION - ,//,5X ,E 1 2.5 
+,3X ,EI2.5,3X ,EI2.5,3X ,E 12. 5, /I 7) 

60 

C 

PRINT 70.aZERO.BXI,PZER02,STDEV2 

61 

70 

FORMATC -,9X .'DATA FUNCTION STATISTICS AFTER',/,IOX 



+‘L IN EAR REGRESSION 8X , 'B Z E R O 1 IX ,'B 1 10X , 

+'V A Rl A NCE',4X, 'STANDARD D E V 1 A T 1 0 N ‘.//.5X ,E12.5 


r 

+,3X ,E 12.5. 3X ,E 12.5, 3X ,E 12.5. /I 7j 

62 

L 

1 F(N PTI ,Ea. 0) GO TO 90 

63 


PRINT 75 

64 

75 

FORMATC ‘,7X ,'T' ,3x.'IN PUT DATA F U N CT 1 0 N ',4X . 
+'LINREG',7X,'STANDARD VARIABLES 1 ./) 

65 


DO 80 1=1, N 

66 

80 

PRINT 85, I * X ( I) ,XNORM( I) ,XSV( I ) 

67 

85 

FORMAT ( . * .18, 3E 18.7/ 

68 

c 

90 

1 F (L 1 N R E G .EO. 3) GO TO MS 

69 


L 1 N REG =LI N REG-1 

70 


P ZERO-PZSRO 1 

71 

r 

JF(LINREG) 125,95.105 

72 

L 

95 

DO 100 1=1. N 

73 

100 

X < I ) =XMDC{ I ) 

74 


P ZER C = P ZER03 

75 

r 

GO TO 125 

76 

V- 

105 

DO 110 1 =1 . N 

77 

110 

X ( 1 ) =X N 0 R M (1) 

78 


P ZER0=P ZE R02 

79 


GO TO 125 


C 
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80 

115 

DO 120 1=1. N 

81 

120 

X (i )=XSV(I) 

82 


PZE R 0 =1 . 

83 

125 

CONTINUE 


c 

C THE INPUT TIME SERIES HAS BEEN PREPROCESSED INTO A 

C CONVENIENT FORM, THE MEM AUTOREGRESSIVE COEFFI- 

C CIENTS ARE NOW COMPUTED. INPUT VARIABLES ARE XCI). 

C PZERO, N,PT, MM, XTDELT, AND FDELT. 


C 

84 CALL ATCREG(X. PZERO. N. PT. M M .XTDELT. FDELT. 

+ PK , A K » K , NPT2 , NPT 3 ) 


85 


86 

87 

88 

89 

90 

91 

92 

93 

94 


c 

C THE AUTOREGRESSIVE COEFFICIENTS, A K (T). HAVE NOW 

C BEEN ESTIMATED USING A LEAST-SQUARE ERROR 

C CRITERION. THESE COEFFICIENTS ARE NOW USED IN THE 

C MEM SPECTRAL ESTIMATOR TO OBTAIN AN ESTIMATE OF THE 

C PGWER SPECTRAL DENSITY FUNCTION. 

C 

CALL MEMSPMC NS ,K , XTDELT , FDELT , PK . A<, S, NM A X , 

+ S8I G,l MA X , N PT 4 ) 

C 

C COMPUTE SPECTRAL AMPLITUDES IN DB AND 

C FREQUENCIES FOR PLOTTING. 

C 

Nmp1=NMAX + 1 
00 499 I — 1 » NMP 1 
L =1-1 

FR(l)=FLOAT(LJ*FDELT 

SPLOT(l)=IO.*ALOulO(S(l)/SBIG) 

STEST =SPLOT(!)+40. 

IF(STEST) 498,498,499 

498 SPLOT ( I )=-40, 

499 CONTINUE 
C 

C *4*4 USER SUPPLIED PLOT ROUTINES **** 

C 38 INPUTS i)a> 

C 

C 

C TIM E VARIABLE. XTIM E(l). 

C INPUT DATA FUNCTION. X (I ). 

C DATA FUNCTION 'WITH DC-VALUE REMOVED, XMDC(I). 

C DATA FUNCTION AFTER LINEAR REGRESSION, XNORMCI). 

C STANDARD V A R I A B L E S.X SV C I ). 

C 

C 33 OUTPUTS 83 

C 

C FREQUENCY VARIABLE IN HERTZ. FR(IJ. 

C POWER SPECTRAL DENSITY FUNCTION. S(l). 

C RELATIVE PSO IN DB. SPLOT(I). 
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95 

96 

97 


98 


99 

100 

101 


102 

103 

104 
i 05 
106 

107 

108 

109 

110 
111 
112 
113 
1 1 4 
115 
1 16 

117 

118 

119 

120 
121 


122 

123 

124 


C 

GO TO 4 
501 STOP 
END 
C 

C THIS SUBROUTINE ESTIMATES THE AUTOREGRESSIVE 

C COEFFICIENTS USING A MAXIMUM ENTROPY CRITERION 

C AND A MINIMUM LEAST SQUARE ERROR. 

C 

SUBROUTINE ATOREGCX, PZERO. N,PT.M M ,XTDELT.FDELT, 

+ PK *AK * K * NPT2 , NPT3) 

C 

DIMENSION A A (100), A KCI00).A KSA V EC 100) 

DIM EN SION B 1C 1000), 92 ( 1000), P{ 1000), X (IOOO) 

C 

INTEGER T ,T P I 
C 

C COMPUTE THE STARTING VALUES, BICT), B2CT), AK(I). 

C AND P(l) FOR THE RECURSIVE EQUATIONS. 

C 

K =1 

8 1(1 ) =X ( I ) 

N M I =N - I 
S2CNM l)=XCN) 

DO 5 T =2.N M I 
8 1 ( T ) =X ( T ) 

5 B 2CT-I ) =X (T) 

XNOM =0. 

X 0 E N =0 . 

NMK=N— K 
00 10 T= 1 , NMK 
XN0M=XNGM+81 (T)*32(T) 

10 XDEN=XDEN+BICT)*SICT)+62CT)*S2(T) 

AK(l)=2.*XNOM/XDEN 
AKSA VE Cl)=AK ( 1} 

P(1 )=PZEP.O*( 1 ♦- AK( 1 ) *AK( 1 ) 

PRINT 15 

15 FORM AT{T.7X,'M EM STARTING V A L U E S:',//.6X ,'K '.8X , 
+'A I C I)-,11X ,'PC I ) ' . 8X , , PCO)' I 5X ,'TOTA L "POW E R',/) 
PRINT 20. K ,A K ( 1),P( 1),PZE R O ,PT 
20 FO R M AT(I 7.E 17.7.3E I 2.3///) 

C 

C COMPUTE A K (T) VALUES USING RECURSIVE EQUATIONS 

C 

25 K =K+ 1 

N M K =N -K 
KM 1 =K - 1 
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125 

126 30 

127 

128 

129 

130 35 

131 

132 

133 

134 

135 40 

136 

137 
135 

139 

140 

14 1 44 

C 

C 

C 

C 

C 

C 


1 42 

45 

143 

46 

144 

50 

145 

51 

146 


147 

52 

1 48 

55 

149 

53 

150 


151 

60 


152 


153 


154 

65 

155 

70 

1 56 

71 

157 

75 


158 

159 

160 .80 

161 85 


00 30 T= 1 ,KM1 
A A (T)=A K (T) 

DO 35 T = 1 »NMK 
T P I =T +1 

d 1 ( T )=6 1 C T)-AA( KM1J *132( T ) 

B2(T)=B2(TPI)-A A (K M l)*al(TPI ) 

XNCM=0. 

X D E N =0 . 

DC 40 T =1 , N M K 

XN0M = XNCM + 3l ( T) *82 ( T) 

XDEN=XDEN+S 1 ( T ) *31 ( T > + 32 £ T ) * 52 ( T ) 

A K( K ) -2 . 4XNQM/X0EN 
AKSAVE) K ) =A K ( K) 

P (X ) =P( KM1 ) * ( 1 . -AK < K ) *AK £ K ) 

DO 45 T=I.KM I 
K M T=K -T 

AK(T)=AA(T)-AK(K)*AA(KMT) 

TO IN H 1 3 IT THE PRINTING OF ALL IMMEDIATE FILTER 
CALCULATIONS AND COEFFICIENTS, CHANGE STATEMENTS 
45, 50, AND 51 TO COMMENTS AND CHANGE THE RANGE 
OF THE DC-LOOP FROM STATEMENT 45 TO STATEMENT 44, 

PRINT 46, K ,T , A K (T ) 

FORMAT ( * \2I8,EI8.7) 

PRINT 51 
FORMAT)///) 

1 F (M M -K )52,55,25 
PRINT 53 

I F(N PT2.EC ,0> GO TO 71 

FORMATC ' 9 10X »' *»* ERROR IN A TO RE G 444*,//) 

PRINT 60 

FORMATC ',7X ,'SU M M A RY OF ITERATIVE kESULTS',//, 
+5X , 'I N D E X ',3X ,'COEFFICI ENT',6X .'ERROR POWER', 

+//.7X ,'K ',8X .'A K (K )',13X ,'PCK )'./) 

PK =P (K ) 

DO 65 1=1, K 

PRINT 70,1 ,A K SA V E (I ),P(I ) 

FORM AT (I 8,3E I 7.7//) 

PRINT 75 , K . P ( K ) 

FORMATCl*, 9XC AUTOREGRESSIVE ESTIMATION SUMMARY 1 , 
+///,8X ,'L E N GTH . K =',13,3X ,'ERROR POWER, P(K):'. 

+E I 3.7.///.SX ,'ORDER'.3X, 

+'AU TO REGRESSIVE COEFFICIENTS',//, 

+8X ,'T',9X ,'A K (T)',//) 

I F ( N PT 3 .EO. 0) GC TO 90 
00 80 T =1 ,M M 
PRINT 85 .T , A K (T ) 

FORMATC '.18.E18.7) 



472 


162 

90 

RETURN 

163 

r 

END 


L 

C 

THIS SUBROUTINE EVALUATES THE MEM SPECTRUM USING 


c 

THE CALCULATED COEFFICIENTS. AK ( T ) * AND THE 


c 

SPECIFIED FREQUENCY INCREMENT. DELTA-F. THE ROUTINE 


c 

CALCULATES THE NY QU 1ST FREQUENCY AND LIMITS THE 


c 

FREQUENCY RANGE ACCORDINGLY. THIS ELIMINATES THE 


c 

c 

POSSIBILITY OF COMPUTING ALIASED SPECTRA, 

1 64 

c 

SUBROUTINE M EMS P M I NS « K , XT DELT , FDEL T , PK , AK , S , NMAX 
+ , S3IG , I MA X i NPT 4 ) 

165 

L 

r 

DIMENSION A K (100).S(1000) 

166 

L 

PI =3.1415926 

167 


NMAX = I NT (1 ,/(2. AXTDELTAFDELT) 

168 


NMP1 =NMAX+ 1 

169 


PRINT 1 

170 

i 

FORMAT (« 1 » ,12X ,' MEM POWER SPECTRAL DENSITY', IX, 
+ ’ESTIMATION' ,///) 

171 


DO 10 1 =1 .N M PI 

172 


L =1-1 

173 


XL -FLOAT (L ) 

174 


PHI=2.*PI*XL*FDELT*XTDELT 

175 


A L P H A =0 . 0 

1 76 


B ETA =0 .0 

177 


DO 5 J=1,K 

178 


ALPHA =AK(J)* COS (J*PHI)+ALPHA 

179 

5 

8cTA = AK'. J)*SIN< J*PHI ) +3ETA 

180 


0=(I.-ALPHA)*(I.-ALPHA)+8ETA*BETA 

181 

10 

S(l)=PK*XTOELT/D 

182 


PRINT 14 

183 

14 

FORMAT (• '.5X.THE LARGEST SPECTRAL COMPONENT IS:', 
+//.7X ,'L',4X , 'FREQUENCY'. 7X.' AM PL ITU OE',/) 

184 


S B 1 G =0 . 0 

185 


00 15 1 =2,N M PI 

186 


XS=S(I ) 

187 

1 5 

IF(XS.GT.SBIG)SBIG=XS 

188 


15=0 

189 

20 

15=1 S +1 

190 


SY EST =S C 1 S ) - S B 1 G 

191 


1 F(STE SY ) 20,21 ,21 

192 

21 

1 M A X =1 S-l 

193 


FREQ=IMAX*FOELT 

194 


PRINT 25.IM AX,FREQ.SBIG 

195 

25 

FORMATe 1 ' . 1 7 , E 12 . 4 . E 1 6 . 4 .///) 

196 


P S =0 . o 
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197 


DO 30 1 =2 .N M A X 

198 

30 

PS=PS+S(I)*FDELT 

199 


PS=PS+S(l)*FDELT/2.+S(NMPI)*FDELT/2e 

200 


PS =2.* P S 

201 


PRINT 31, K .PS 

202 

31 

FORM AT(' 5X , 'THE TOTAL POWER IN THE ESTIMATEO'. 

+ 1X, 'SPECTRUM IS' .//,6X, LENGTH, K =' ,13, 5X . 

+ • POWE R= ' ,E13. 7, /////) 

203 

C 

1 F ( N PT 4 .EO. 0 ) GO Te 45 

204 


PRINT 35 

205 

35 

FORM AT(' '.6X , 'L ’ ,6X ,' FREQ-HZ 1 , 6X. 'NORMAL 1 ZED' , 

+3X. TOTAL POWER' .3X,'PELATIVE',/,27X, 'SPECTRAL', 

+ 1 9 X» * AM PL ITUDE't / »27X« 'DENSITY* » 7X» ’OB' * 1 3 X ♦ 'DB' »/) 

206 


DO 40 1=1, N M PI 

207 


L =1-1 

208 


FRSQ = L*FO£LT 

209 


PS1=S( 1 ) *FO£LT 

210 


PSI=10.*ALOGI0(PSI) 

211 


PS2 = S< i ) /SB I G 

212 


PS2= 1 0. 4AL0G1 0( »S2 ) 

213 

40 

PRINT 41.L.FREC.S(I),PS1.PS2 

2 1 4 

41 

FORMAT ( ' -.I7.2E 1 3 . 3 . F 1 1 . 1 , F 1 4 . 1 ) 

215 

45 

RETURN 

2 16 


.END 


MAXIMUM ENTROPY SPECTRAL ANALYSIS 


PTOPT 

LINREG 

NS 

MM 

DELTA-T. SEC 

DELTA-F. HZ 

1111 

3 

41 

15 

0.6000000E 02 

0.2500000E -04 


INPUT DATA FUNCTION STATISTICS: HER 

AVERAGE MEAN-SQUARE VARIANCE 

0.82960E 02 0.31102E 05 0.2462EE 05 


A 7/29/75 

STANDARD DEVIATION 
0.15756E 03 


DATA FUNCTION STATISTICS AFTER 
LINEAR REGRESSION: 
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8ZERO El VARIANCE STANDARD DEVIATION 

- 0 . 88 342 E 02 0.13595E 02 0.15376E 05 0.12360E 03 


T INPUT DATA FUNCTION LINREG STANDARD VARIABLES 


1 

-0 . 63925 99E 

02 

0.1625516E 

02 

0.1315165E 

00 

2 

-0. 1643800E 

03 

-0.9235213E 

02 

-0.7471983E 

00 

3 

-0.2860801E 

03 

- 0. 2222095E 

03 

-0. 1797842E 

01 

4 

-0.3361799E 

03 

-0 * 2 804666E 

03 

-0.2269185E 

0 1 

5 

-0.2902000E 

03 

-0.2426439E 

03 

-0. 1963172E 

01 

6 

-0. 1010600E 

03 

-0.6166125E 

02 

-0.4988860E 

00 

7 

0.8405000E 

02 

0.1152915E 

03 

0.9327948E 

00 

8 

0. 1956200E 

03 

0.2187041E 

03 

0.1769481E 

01 

9 

0.2508800E 

03 

0.2658066E 

03 

0.2150576E 

01 

10 

0.2205500E 

03 

0.2273195E 

03 

C.1839186E 

01 

11 

0. 1111300E 

03 

0.1 09 7 423 E 

03 

0.S878980E 

00 

12 

-0 « 2 083 0 00E 

02 

-0.3037497E 

02 

-0.2457564E 

00 

1 3 

-0.6872000E 

02 

-0.8642226E 

02 

-0.6992212E 

00 

1 4 

-0.3719000E 

02 

-0.6304955E 

02 

-0.51 01 1 84E 

00 

15 

0.7963000E 

02 

0.4561317E 

02 

0.3690450E 

00 

16 

0 . 1437900E 

03 

0.1016159E 

03 

0.8221490E 

00 

17 

0. 1380600E 

03 

0.8772861E 

02 

0.7097906E 

00 

18 

0.1957700E 

03 

0.1372812E 

03 

0.11 10708E 

01 

19 

0. 1478500E 

03 

0.8120407E 

02 

0.6570022E 

CO 

20 

0.5160001£ 

02 

- 0 . 2320322E 

02 

-0.1877315E 

00 

21 

0. 1438000E 

02 

-0.6858051E 

02 

- 0. 5548680E 

00 

2 2 

-0.3524001E 

02 

-0.1263578E 

03 

-0.1022329E 

01 

23 

0.4507001E 

02 

- 0. 5420503E 

02 

-0 9 4385596E 

00 

24 

0 . 753 1000E 

02 

-0.3212231E 

02 

-0.2598937E 

00 

25 

0.2137500E 

03 

0 . 98 1 6 025 E 

02 

0.7941904E 

00 

26 

0 C2970300E 

03 

0.1732830E 

03 

0.1401990E 

01 

27 

0.2561599E 

03 

0.1242556E 

03 

0.1005322E 

01 

28 

0.15541 OOE 

03 

0. 1534859E 

02 

0.1241816E 

00 

29 

0.7478000E 

02 

-0.7343871E 

02 

-0.5941745E 

00 

30 

0.4120000E 

02 

-0.11 5 1760E 

03 

-O.9318605E 

00 

31 

0.6267000E 

02 

*0 . 1 01 8633E 

03 

-0.8241 505E 

00 

32 

0. 1524000E 

03 

-0.2029053E 

02 

-0. 1641656E 

00 

33 

0.2246000E 

03 

0.4375220E 

02 

0.3539882E 

00 

34 

0.2379500E 

03 

0 .4894458E 

02 

0.3959985E 

00 

3S 

0.241*1 OOE 

03 

0 .4424756E 

02 

0.3579961E 

00 

36 

0.1943700E 

03 

-0.1094971 E 

02 

-0.8859134E - 

Ol 

37 

0. 1155000E 

03 

-0.9797690E 

02 

-0.7927070E 

00 

38 

0.9912000E 

02 

-0.1225141 E 

03 

-0.9912317E 

■00 

39 

0. 1357500E 

03 

- 0. 9404160E 

02 

-0. 7608674E 

00 

40 

0.2149100E 

03 

-0.2303882 E 

02 

-C. 1864014E 

00 

41 

0. 3344900E 

03 

0.8838379E 

02 

0.715091 5E 

00 



TOTAL 
0.311E 

1 OM426 599F 01 


A I ( 1 J P( 1 ) P (O ) 

0. 7703549c 00 0.407E 00 O.LOOE 01 


1 0.1273918E 01 

2 -0.S961S46E 00 


1 0. 1286666E 01 

2 -0.5537803E 00 

3 -0.2698378E 00 


1 0. 1269886E 01 

2 -0 « 4901 22 IE CO 

3 -0 #1391 94 1 E 00 

4 -0.2324146E 00 


1 0.1255294E 01 

2 -0.4757471E 00 

3 -0.1 305848E 00 

4 -0.2 02 1002 E 00 

5 0.1573693E 00 


1 0 . 1249800E 01 

2 -0 « 4897 244 E 00 

3 -0.1 126345E 00 

4 -0.1905018E 00 

5 O . 1996245 E 00 

6 -0.4964290E -O I 


POWER 

05 
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8 

1 

0.1277999E 

01 

8 

2 

-0. 5054857E 

00 

8 

3 

- 0.492 5478 E - 

Ol 

8 

4 

-0.2509851E 

00 

8 

5 

o . 1638636E 

00 

8 

6 

-0.2051279E 

00 

8 

7 

0.4856238E 

00 


9 

1 

0. 1214496E 

01 

9 

2 

-0.4083553E 

00 

9 

3 

-0.9026274E - 

-Ol 

9 

4 

-0.2182105E 

00 

9 

5 

0. 1 136636E 

00 

9 

6 

-0.2149794E 

00 

9 

7 

0*3 84520 7E 

00 

9 

a 

-0.6188023:- 

01 


10 

1 

0.1196949E 

01 

10 

2 

-0.4137836E 

00 

10 

3 

-0.5655117E 

-Ol 

10 

4 

-0.2370692E 

00 

10 

5 

0. 1236346E 

00 

10 

6 

-0.2341216E 

00 

10 

7 

0.3766007E 

00 

10 

8 

-0.9770262E 

-Ol 

10 

9 

-0.9347177E 

-Ol 


1 1 

1 

0.1195743E 01 

11 

2 

-0.4150690 E 00 

11 

3 

-0. 57894-82E -0 1 

1 1 

4 

-0.2318900E 00 

11 

5 

0 . 1204148E 00 

11 

6 

-0.232421 3E 00 

11 

7 

0.3 733404E 00 

1 1 

8 

-0. 9848028E -0 1 

1 1 

9 

-0 .9916228E -0 1 

1 1 

10 

-0.7126266E -O 1 
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12 

1 

0. 1195573E 

01 

12 

2 

-0.4159462E 

00 

12 

3 

-0.5911542E 

-Ol 

1 2 

4 

-0.2331021E 

00 

12 

5 

O .1250103E 

00 

12 

6 

-0.2352822E 

00 

1 2 

7 

0.3748226E 

00 

12 

8 

-0* 1013346E 

00 

12 

9 

-0.9987491E 

-Ol 

12 

10 

-0.7637179E 

-Ol 

12 

1 1 

0.9661391E 

-03 


13 

1 

0. 1 195349E 

01 

1 3 

2 

-0.4159285E 

00 

13 

3 

-0.6050623E 

-Ol 

13 

4 

-0.2349269E 

00 

13 

5 

0 • 1 231 648E 

00 

13 

6 

-0.2284563E 

00 

13 

7 

0.3705379E 

00 

13 

8 

-0.9905803E 

-Ol 

13 

9 

-0 . 1041 199 E 

00 

13 

10 

-0.7 74483 1 E 

-Ol 

13 

11 

-O .6608676E 

-02 

13 

12 

0. 94634856 

-02 


1 4 

1 

0.1198529E 

01 

1 4 

2 

-0.4175817E 

00 

1 4 

3 

-O .5935176E 

-Ol 

1 4 

4 

-0.Z213914E 

00 

i 4 

5 

0.1413535E 

00 

1 4 

6 

-0.2111518E 

00 

1 4 

7 

0.3058087E 

00 

1 4 

8 

-0. 59 14903E 

- 01 

1 4 

9 

-0.1256355E 

00 

1 4 

10 

-0.3641000E 

-Ol 

1 4 

11 

0.3961150E 

-02 

1 4 

12 

0.8212191E 

-Ol 

1 4 

13 

-0. 22702"63E 

00 


15 

1 

0. 1184735E 

01 

15 

2 

- O .3996562 E 

00 
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15 

3 

-0.6583589E -O 1 

15 

4 

-0.2217042E 00 

15 

5 

0 • 1 442283E 00 

15 

6 

-0.2012320E 00 

15 

7 

0.3104789E 00 

15 

8 

-0. d329493E - 01 

15 

9 

-0. 1089634E 00 

15 

■ /-> 
i W 

-O.4757093E-OI 

15 

1 1 

0.2 144169E - 0 1 

15 

12 

0.5680815E -01 

1 5 

13 

-0.194C551E 00 

15 

14 

0 .8005691E -0 1 


SUMMARY OF ITERATIVE RESULTS 


NO EX 

COEFF IC t ENT 

ERROR POWER 

K 

AK(K) 

P ( K) 

1 

0 « 7 703 54 9 E 00 

O.4065534E 00 

2 

-0.851d724E 00 ' 

0. 1115230E 00 

3 

-0.1792291E 00 

0. 1079406E 00 

4 

0.7112604E -O 1 

0.1 0 73945E 00 

5 

0.2359126E 00 

O.1014174E 00 

6 

O .6185063E -O 1 

0 . 1010294E 00 

7 

0.8881807E -01 

0.1002324E 00 

8 

-0.31 74950E 00 

0 .9012860E -0 1 

9 

-0.2000117E 00 

0 .8652300E -0 1 

10 

-0.S 772367E -O 1 

0 .8585715E -0 1 

1 1 

-0 . 1375243E -0 1 

0.8 5840a8E -0 1 

12 

-0.1230915E-01 

0.8582783E -0 1 

13 

-0.1821105E -O 1 

0 .8579934E -0 1 

14 

0.1 746899E 00 

O.8318102E-OI 

15 

0 .7895762E -0 1 

0 .826624C E -0 1 
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AUTOREGRESSIVE ESTIMATION SUMMARY 


LENGTH, K = 15 ERROR POWER > P (K ) =0 .8266240E -0 I 


ORDER AUTOREGRESSIVE COEFFICIENTS 
T AK { T) 


1 0 « 1 1 34 7 35 E 01 

2 -0.3996562E 00 

3 -0.65S3589E -O I 

4 -O.2217042E 00 

5 0 . 14 42 2 83E 00 

6 -0.2012320E 00 

7 0.3104789E 00 

8 -0.8329493E-0I 

9 -0. 10S9634E 00 

10 -0.4757093E-OI 

11 0.2144169E-0I 

12 O .8680815E -O I 

13 -O.1940551E 00 

14 O.8005691E-OI 

15 0.7895762E-0I 


MEM POWER SPECTRAL DENSITY ESTIMATION 


THE LARGEST SPECTRAL COMPONENT IS: 
L FREQUENCY AMPLITUDE 

77 0.1925E-02 0.4210E 04 


THE TOTAL POWER IN THE ESTIMATED SPECTRUM IS 


LENGTH. K= 15 


P0#ER=0,998S377E 00 
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FREQ-HZ 

NORMALIZED 

TOTAL POWEP 

RELATIVE 


SPECTRAL 


AM PI .ITU D E 


DENSITY 

DB 

DB 


0 O.OOOE 00 0.287E 

1 0.25CE-C4 0.287E 

2 0 .SO 0 E -04 0.286E 

3 0.750E-04 0.285E 

4 0.100E-03 C.284E 

5 O.12SE-03 0.282E. 

6 0.150E-03 C.280E 

7 O.17SE-03 0.278E 

8 0.200E -03 0.277E 

9 O.225E-03 0.275E 

10 O.2S0E-03 0.274E . 

M O.275E-03 0.273E 

12 0.3COE-03 0.273E 

13 0.325E -03 0.274E 

14 0.350E-03 0.276E 

15 O.37SE-03 0.279E 

16 0.400E-03 0.283E 

17 0.4256-03 0.288E 

18 0.4506-03 0.295E 

19 O.475E-03 0.304E 

20 0.500E-03 0.315E 

21 0.525E-03 0.328E 

22 0.550E-03 0.344E 

23 0.S75E-03 0.363E 

24 0.600E-03 0.387E 

25 O.625E-03 0.415E 

26 0.650E-03 0.449E 

27 0.675E -03 0.490E 

28 0.700E-03 0.539E 

29 O.72SE-03 0.599E 

30 0.750E -03 0.672E 

31 O.775E-03 0.761E 

32 O.8COE-03 C.869E 

33 0.82SE-03 O.IOOE 

34 0.8S0E -03 0.116E 

35 0.S7SE -03 0.134E 

36 0.900E-03 0.154E 

37 O.92SE-03 0.175E 

38 0.950E-03 0.193E 

39 O.97SE-03 0.205E 

40 0.100E-02 0.209E 

41 0.102E-02 O.203E 

42 0.10SE-02 0.189E 

43 0.108E-02 0.171E 

44 0.110E-02 O.ISIE 

O.112E-02 0.133E 


02 

-31.4 

-21.7 

02 

-31.4 

-21.7 

02 

-31.5 

-21.7 

02 

-31.5 

-21.7 

02 

-31.5 

-21.7 

02 

-31.5 

-21.7 

02 

-31.5 

-21.8 

02 

-31.6 

-21.8 

02 

-31.6 

-21.8 

02 

-31.6 

-21.8 

02 

-31.6 

-21.9 

02 

-31.7 

-21.9 

02 

-31.7 

-21.9 

02 

-31. .6 

-21.9 

02 

-31.6 

-21.8 

02 

-31.6 

-21.8 

02 

-31.5 

-21.7 

02 

-31 .4 

-21.6 

02 

-31.3 

-21.5 

02 

-31.2 

-21.4 

02 

-31.0 

-21.3 

02 

-30.9 

-21.1 

02 

-30.7 

-20.9 

02 

-30.4 

-20.6 

02 

-30. 1 

-20.4 

02 

-29.8 

-20.1 

02 

-29.5 

-19.7 

02 

-29.1 

-19.3 

02 

-28.7 

-18.9 

02 

-28.2 

-18.5 

02 

-'ll .1 

-18.0 

02 

-11 .1 

-17.4 

02 

-26.6 

-16.9 

03 

-26.0 

-16.2 

03 

-2S.4 

-15.6 

03 

-24.8 

-15.0 

03 

- 24.1 

- 14.4 

03 

-23.6 

-13.8 

03 

-23.2 

-13.4 

03 

-22.9 

-13.1 

03 

-22.8 

-13.0 

03 

-22.9 

-13.2 

03 

-23.3 

-13.5 

03 

-23.7 

-13.9 

03 

-24.2 

-14*4 

03 

-24.8 

-15.0 


45 
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46 

C.I15E-OZ 

0.116E 

47 

0.117E-02 

0.102E 

48 

0.120E-02 

0.908E 

49 

0.123E -02 

0.813E 

50 

G * 1 25E-C2 

0.735E 

51 

0. 12 7 E - 02 

0.672E 

52 

0.130E-OZ 

0.622E 

53 

0.132E-02 

0.581E 

54 

0.135E-02 

0.550E 

55 

0.137E-02 

0.527E 

56 

0.140E-02 

0.510E 

57 

0.142E-02 

0.499E 

58 

0.145E-02 

0 .495 E 

59 

0.148E-02 

0.496E 

60 

0.150E-02 

0.503E 

61 

0.152E-02 

0.517E 

62 

0.155E -02 

0.538E 

63 

0.157E-02 

0.569E 

64 

0.160E-02 

O.610E 

65 

0.163E-02 

0.666E 

66 

0.165E-02 

0.741E 

67 

0.167E -02 

0.842E 

68 

0.170E -02 

a.981E 

69 

0.172E-02 

0.118E 

70 

0.175E-02 

0.146E 

71 

0.178E-02 

0.191E 

72 

0. 180 E - 02 

0.264E 

73 

0.182E-02 

0.398E 

74 

0.185E-02 

0.677E 

75 

0.188E-02 

0.138E 

76 

0.190E-02 

0.338E 

77 

0.192E-02 

0.42 IE 

78 

0. 195 E - 02 

0.171E 

79 

0.197E-02 

0.734E 

80 

0.200E -02 

0.385E 

81 

0.203E -02 

0.232E 

82 

0.205E -02 

0.154E 

83 

0.207E-02 

0.109E 

84 

0.210E -02 

0.812E 

85 

0.212E -02 

C.628E 

86 

0.2 15E-02 

O.501E 

87 

O.218E-02 

C.410E 

88 

0 .220E-02 

0.342E 

89 

0.222E-02 

0.291E 

90 

0 .225E-02 

C.251E 

91 

0.228E -02 

O.220E 

92 

0.230E-02 

0.195E 

93 

0.233E -02 

0.175E 

94 

0.235E-02 

0.158E 

95 

0.237E-02 

0. 144E 


03 

- 25.4 

- 15.6 

03 

- 25 . 9 ' 

- 16.1 

02 

-26.4 

- 16.7 

02 

- 26.9 

- 17.1 

02 

-27.4 

- 17.6 

02 

- 27.7 

- 18.0 

02 

- 28.1 

- 18.3 

02 

- 28.4 

- 18.6 

02 

- 28.6 

- 18.8 

02 

- 28.8 

- 19.0 

02 

- 28.9 

- 19.2 

02 

- 29.0 

- 19.3 

02 

- 29.1 

- 19.3 

02 

- 29.1 

- 19.3 

02 

- 29.0 

- 19.2 

02 

- 28.9 

- 19.1 

02 

- 28.7 

- 18.9 

02 

- 28.5 

- 18.7 

02 

- 28.2 

- 18.4 

02 

- 27.8 

- 18.0 

02 

- 27.3 

- 17.5 

02 

- 26.8 

- 17.0 

02 

- 26.1 

- 16.3 

03 

- 25.3 

- 15.5 

03 

- 24.4 

- 14.6 

03 

- 23.2 

- 13.4 

03 

- 21.8 

- 12.0 

03 

- 20.0 

- 10.2 

03 

- 17.7 

- 7.9 

04 

- 14.6 

- 4.8 

04 

- 10.7 

- 0.9 

04 

- 9.8 

0.0 

04 

- 13.7 

- 3.9 

03 

- 17.4 

- 7.6 

03 

-20.. 2 

- 10.4 

03 

- 22.4 

- 12.6 

03 

-24.1 

-14.4 

03 

- 25.6 

- 15.9 

02 

- 26.9 

- 17.1 

02 

- 28.0 

- 18.3 

02 

- 29.0 

- 19.2 

02 

- 29.9 

- 20.1 

02 

- 30.7 

- 20.9 

02 

-31 .4 

- 21.6 

02 

- 32.0 

- 22.2 

02 

- 32.6 

- 22.8 

02 

-33 . 1 

- 23.3 

02 

- 33.6 

- 23.8 

02 

- 34.0 

-24.3 

02 

-34.4 

- 24.7 
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96 

0.240E-02 

0.133S 

97 

0 .243E-02 

0.123E 

9S 

0.245E-02 

0.115E 

99 

0 .247E-02 

0.10SE 

100 

0.250E-02 

O.IOIE 

101 

0.252E -02 

C .962 E 

102 

0.255E-02 

C.916E 

103 

0 .25SE-02 

0.S76E 

104 

0.260E -02 

0.S40E 

105 

0.262E -02 

O.SOSE 

106 

0.265E -02 

0.779E 

107 

0.268E -02 

0.7S2E 

IOS 

0.270E-02 

0.727E 

109 

0.273E -02 

0.704E 

110 

0.275E-02 

0.681E 

111 

0.277E -02 

0.659E 

112 

0.280E -02 

0.637E 

113 

O.283E-02 

0.616E 

1 14 

0.285E -02 

0.594E 

115 

0.2.S8E-02 

0.572E 

116 

0.290E -02 

C.SSOE 

117 

0.292E -02 

0.S28E 

118 

0.295E -02 

0.505E 

1.19 

0.29SE -02 

0.483E 

120 

0.300E-02 

0.461E 

121 

0.302E -02 

0.439E 

122 

0.305E -02 

0.418E 

123 

0.308E -02 

0.398E 

124 

0.310E-02 

0.378E 

125 

0.313E-02 

0.359E 

126 

0.315E-02 

C.341E 

127 

0.317E-02 

0* 32 4E 

128 

0.320E -02 

0.308E 

1 29 

0.323E -02 

0.294E 

130 

0.325E-02 

C.280E 

131 

0.328E-02 

0.26SE 

132 

0.330E -02 

0.256E 

133 

0.332E-02 

0.246E 

134 

0.33SE-02 

0.237E 

13S 

0.338E -02 

0.229E 

136 

0.340E -02 

0.221E 

137 

0.343E-02 

0.2.ISE 

138 

0.34SE-02 

0.209E 

139 

0.348E-02 

0.20SE 

140 

0.3S0E-02 

0.201E 

141 

0 .353E-02 

0.198E 

142 

0.3SSE -02 

0.195E 

143 

0.3S7E-02 

0.194E 

144 

0.360E-02 

0.193E 

145 

0.363E-02 

0.193E 


02 

-34.8 

-2S.0 

02 

-35.1 

-25.3 

02 

-35.4 

-25.7 

02 

-35.7 

-25.9 

02 

-36.0 

-26.2 

01 

-36.2 

-26.4 

01 

-36.4 

-26.6 

01 

-36.6 

-26.8 

01 

-.36.8 

-27.0 

01 

-36.9 

-27.2 

01 

-37.1 

-27.3 

01 

-.37.3 

-27.5 

01 

-37.4 

-27.6 

01 

-37. S 

-27.8 

01 

-.37.7 

-27.9 

01 

-37. S 

-2S.1 

01 

-3S.0 

-28.2 

01 

-3S.1 

-28.3 

01 

-3S.3 

-2S.5 

01 

-38.4 

-2S.7 

01 

-38.6 

-2S.S 

01 

-38.8 

-29.0 

01 

-39.0 

-29.2 

01 

-39.2 

-29.4 

01 

-39.4 

-29.6 

01 

-39.6 

-29. S 

01 

-39. S 

-30.0 

01 

-40.0 

-30.2 

01 

- 4 0 « 2 

-30.5 

01 

-40.5 

— o 0 © * 

01 

l 

o 

. 

->i 

-30.9 

01 

-40.9 

-31.1 

01 

-41.1 

-31.4 

01 

-41 .3 

-31.6 

01 

-41.5 

-31.8 

01 

-41 .7 

-32.0 

01 

-41 .9 

-32.2 

01 

-42.1 

-32.3 

01 

-42.3 

-32.5 

01 

-42.4 

-32.7 

01 

-42.6 

-32.8 

01 

-42.7 

-32.9 

01 

— 42 . 8 

-33.0 

01 

-42.9 

-33.1 

01 

-43.0 

-33.2 

01 

-43.1 

-33.3 

01 

-43 .1 

-33.3 

01 

-43*1 

-33.4 

01 

-43 .2 

-33.4 

01 

-43.2 

-33.4 
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1 4-6 

0.365E -02 

0.194E 

147 

0.368E -02 

0.196E 

148 

0.370E -02 

0.199E 

149 

0.372 E -02 

0.202E 

150 

0.375E -02 

0.207E 

151 

0.378E -02 

0.213E 

152 

0.380E -02 

0.220E 

153 

0.383E -02 

0.229E 

154 

0.385E -02 

0.239E 

155 

0.388E -02 

O.250E 

156 

0.390E -02 

0.264E 

157 

0.392E -02 

0.280E 

158 

0.395E -02 

0.298E 

159 

0.397E -02 

0.319E 

160 

0 .400E-02 

0.343E 

161 

0.402 E -02 

0.369E 

162 

O.405E-02 

0.399E 

163 

0.407E -02 

0 .432 E 

164 

0.410E -02 

0.466E 

"•65 

0.41 2£ — 02 

O.502E 

166 

0.415E -02 

0.538E 

167 

0.417E-02 

0.570E 

168 

0.420E-02 

0.597E 

169 

O.422E-02 

0.614E 

170 

0.425E -02 

0.620E 

171 

0.42 7 E - 02 

0.613E 

172 

0.430E -02 

0.594E 

173 

0.432E -02 

0.564E 

174 

0.435E-02 

0.527E 

175 

0 .437E-02 

0.486E 

176 

0.440E-02 

0.444E 

177 

0 » 442E-02 

0.4 03E 

178 

0 .445E-02 

0.364E 

179 

0.447E-02 

0.329E 

180 

0.450E-02 

0.298E 

181 

0.452E-02 

O.270E 

182 

0.455E -02 

0.246E 

183 

0.457E -02 

0.225E 

184 

0.460E-02 

O.206E 

185 

0.463E -02 

0.19OE 

186 

0.465E -02 

0.176E 

187 

0.467E-02 

0.164E 

188 

0.470E -02 

0.153E 

189 

0.472E-02 

0.144E 

190 

O.475E-02 

0.136E 

191 

0.477E-02 

0.129E 

192 

0.480E -02 

0.123E 

193 

0 .482 E - 02 

O.II8E 

194 

O.485E-02 

0.114E 

195 

0.487E-02 

0.11DE 


01 

-43.1 

- 33.4 

01 

- 43.1 

-33.3 

01 

-43.0 

-33.3 

01 

-43.0 

-33.2 

01 

-42 .9 

-33.1 

01 

— 42 « 7 

-33.0 

01 

-42.6 

-32.8 

01 

- 42.4 

-32.7 

01 

-42.2 

-32.5 

01 

- 42.0 

- 32.3 

01 

- 41.8 

-32.0 

01 

-41 .5 

-31.8 

01 

-41 .3 

-31.5 

01 

- 41 . 0 

-31.2 

01 

-40.7 

-30.9 

A * 
V 1 

- 40.3 

-30.6 

01 

-40.0 

-30.2 

01 

-39.7 

-29.9 

01 

-39.3 

-29.6 

01 

-39.0 

-29.2 

01 

-38.7 

-28.9 

01 

-38.5 

-28.7 

01 

-38.3 

-28.5 

01 

-38.1 

- 28.4 

01 

-38.1 

-28.3 

01 

-38.1 

- 28.4 

01 

-38.3 

-28.5 

01 

-38.5 

-28.7 

01 

-38.8 

-29.0 

ox 

-39.2 

- 29.4 

01 

-39.5 

-29.8 

01 

-40.0 

-30.2 

01 

- 40.4 

-30.6 

01 

- 40.8 

-31.1 

01 

- 41.3 

-31.5 

01 

- 41.7 

-31.9 

01 

- 42 . 1 

-32.3 

01 

- 42.5 

-32.7 

01 

- 42.9 

-33.1 

01 

- 43.2 

-33.5 

01 

- 43.6 

-33.8 

01 

-43.9 

-34.1 

01 

- 44.2 

- 34.4 

01 

-44.4 

-34.7 

01 

- 44.7 

- 34.9 

01 

- 44.9 

-35.1 

01 

- 45.1 

-35.3 

01 

- 45.3 

-35.5 

01 

- 45.5 

-35.7 

01 

-45.6 

-35.8 
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196 

0 .4 90E - 02 

0.107E 

197 

0.492E-02 

0.1 04E 

198 

0 .4 95E — 02 

0.102E 

199 

0.497E -02 

0.101 E 

200 

0.500E -02 

0.993E 

201 

0.502E -02 

0.984E 

202 

0.505E -02 

C.978E 

203 

0.508E -02 

0.976E 

204 

0.510E-02 

C.977E 

205 

0.512E -02 

0.9SIE 

206 

0.515E -02 

0.9SSE 

207 

0 .517E-02 

C.99SE 

208 

0.520E -02 

O.IOI E 

209 

0.522E-02 

0.103E 

210 

0.525E -02 

0.1 04E 

211 

0.527E -02 

O.106E 

212 

0.530E-02 

O.IOSE 

213 

0.532E -02 

0.111E 

214 

0.53SE -02 

0.113E 

215 

0.537E -02 

0.115E 

216 

0.540E-02 

0.11SE 

217 

0.542E-02 

0. 12 IE 

218 

0.545E-02 

0.123E 

219 

0.547E-02 

0.125E 

220 

0.550E-OZ 

0.127E 

221 

0.552E -02 

0.129E 

222 

0. £555-02 

C.131E 

223 

0.557E-02 

0.132E 

224 

0.560E-02 

0.132E 

225 

0.562E-02 

0.132E 

226 

0.565E -02 

0.132E 

227 

0.567E-02 

0.131E 

228 

0.570E -02 

0.130E 

229 

0.573E -02 

0.129E 

23 0 

0.575E-02 

0.127E 

231 

0.S77E -02 

0.125E 

23 2 

0.580E -02 

0.123E 

233 

0.582E-02 

0.120E 

234 

0.585E-02 

0.118E 

23S 

O.S87E-02 

0.116E 

236 

0.590E -02 

C.113E 

237 

0.592E -02 

O.IIIE 

238 

0.595E-02 

0.109E 

239 

0.597E-02 

0.107E 

240 

0.600E-02 

O.IOSE 

241 

0.602E-02 

0.103E 

242 

0.605E -02 

0. 102 E 

243 

0.607E-02 

O.IOIE 

244 

0.610E -02 

C.IOOE 

245 

0.612E -02 

0.997E 


01 

- 45.7 

-35.9 

01 

- 45.8 

-36.1 

01 

-45.9 

-.36.1 

01 

- 46.0 

-36.2 

00 

-46.1 

-36.3 

00 

-46.1 

-36.3 

00 

-46.1 

-36.3 

00 

- 46.1 

-36.3 

00 

- 46 . 1 

-36.3 

00 

- 46.1 

-36.3 

00 

- 46.1 

-36 *3 

00 

- 46.0 

-36.3 

01 

- 46.0 

-36.2 

01 

- 45.9 

-36.1 

01 

- 45.8 

-36.1 

01 

- 45.8 

-36.0 

01 

- 45.7 

-35.9 

01 

- 45.6 

-.35.8 

01 

- 45.5 

-35.7 

01 

- 45.4 

-35.6 

01 

-45 .3 

-35.5 

01 

-45.2 

-35.4 

01 

- 45.1 

-35*3 

01 

- 45.0 

-35*3 

01 

- 45.0 

-35.2 

01 

- 44.9 

-35.1 

01 

- 44.9 

-35.1 

01 

- 44.8 

-.35.1 

01 

-44 .8 

-35.0 

01 

- 44.8 

-35.0 

01 

- 44.8 

-35.0 

01 

- 44.8 

-35.1 

01 
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